Analysis and computation of an optimality equation arising in an impulse control problem with discrete and costly observations

Abstract The framework of discrete and costly observations provides an attractive mathematical tool for stochastic control of biological and ecological systems in natural environment. However, this framework is still germinating, and its applications in these research areas are still rare. In this framework, the system dynamics to be controlled is continuous-time, while the observations by the decision-maker are discrete-time. The control problem of the system dynamics has a time-hybridized type nature, by which the optimality equation has a different form from the conventional ones. The objectives as well as contributions of this paper are formulation of the optimality equation focusing on population management problems, its detailed mathematical analysis, numerical discretization, and a demonstrative application. We show that the optimality equation is solvable uniquely and its solution can be approximated recursively. A numerical method to discretize the optimality equation is established with a simple finite difference scheme and the recursive formula. Mathematical analysis results such as monotonicity, stability, and convergence of the numerical method are discussed as well. The present model is then applied to a population management problem. Finally, an advanced problem subject to model ambiguity is also analyzed.

[1]  R. Lande,et al.  Stochastic Population Dynamics in Ecology and Conservation , 2003 .

[2]  Amiya Ranjan Bhowmick,et al.  A simple approximation of moments of the quasi-equilibrium distribution of an extended stochastic theta-logistic model with non-integer powers. , 2016, Mathematical biosciences.

[3]  P. Kumam,et al.  Hybrid iterative scheme by a relaxed extragradient method for solutions of equilibrium problems and a general system of variational inequalities with application to optimization , 2009 .

[4]  L. Allen.  An Introduction to Stochastic Epidemic Models , 2008 .

[5]  Huyên Pham,et al.  Robust Markowitz mean‐variance portfolio selection under ambiguous covariance matrix , 2016, 1610.06805.

[6]  Jinbiao Wu,et al.  Optimal exchange rates management using stochastic impulse control for geometric Lévy processes , 2019, Math. Methods Oper. Res..

[7]  A. Zeeuw,et al.  Managing a renewable resource facing the risk of a regime shift in the ecological system , 2017 .

[8]  Tak Kuen Siu,et al.  Optimal insurance risk control with multiple reinsurers , 2016, J. Comput. Appl. Math..

[9]  Ke Wang,et al.  Stability of stochastic Richards growth model , 2015 .

[10]  L. Tsimring.  Noise in biology , 2014, Reports on progress in physics. Physical Society.

[11]  Dariusz Gatarek,et al.  Ergodic impulsive control of Feller processes with costly information , 1990 .

[12]  Liu Yang,et al.  An inverse problem arisen in the zero-coupon bond pricing ☆ , 2010 .

[13]  C. Jang,et al.  Assessment of spatiotemporal variations in river water quality for sustainable environmental and recreational management in the highly urbanized Danshui River basin , 2019, Environmental Monitoring and Assessment.

[14]  Dusan M. Stipanovic,et al.  Guaranteed Collision Avoidance with Discrete Observations and Limited Actuation , 2013 .

[15]  A. Xepapadeas,et al.  Climate engineering under deep uncertainty , 2018, Journal of Economic Dynamics and Control.

[16]  H. Yoshioka,et al.  Optimal harvesting policy of an inland fishery resource under incomplete information , 2019, Applied Stochastic Models in Business and Industry.

[17]  Ke Wang,et al.  A note on stability of stochastic logistic equation , 2013, Appl. Math. Lett..

[18]  P. Forsyth,et al.  Numerical Methods for Nonlinear PDEs in Finance , 2012 .

[19]  Erhan Bayraktar,et al.  Convergence of Implicit Schemes for Hamilton-Jacobi-Bellman Quasi-Variational Inequalities , 2017, SIAM J. Control. Optim..

[20]  B. Øksendal,et al.  Applied Stochastic Control of Jump Diffusions , 2004, Universitext.

[21]  Paul C. Fife,et al.  Second-Order Equations With Nonnegative Characteristic Form , 1973 .

[22]  Wei Zhu,et al.  Maximizing a robust goal-reaching probability with penalization on ambiguity , 2019, J. Comput. Appl. Math..

[23]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[24]  Huyen Pham,et al.  Continuous-time stochastic control and optimization with financial applications / Huyen Pham , 2009 .

[25]  Sabyasachi Bhattacharya,et al.  A new growth curve model for biological growth: some inferential studies on the growth of Cirrhinus mrigala. , 2014, Mathematical biosciences.

[26]  F. Viens,et al.  Optimal reinsurance and investment strategies for insurers with mispricing and model ambiguity , 2017 .

[27]  Amiya Ranjan Bhowmick,et al.  Physiological responses of fish under environmental stress and extension of growth (curve) models , 2017 .

[28]  Ricardo Castro-Santis,et al.  An impulsive fishery model with environmental stochasticity. Feasibility. , 2016, Mathematical biosciences.

[29]  A. Skonhoft,et al.  New insights from the canonical fisheries model – Optimal management when stocks are low , 2018, Journal of Environmental Economics and Management.

[30]  Mini Ghosh,et al.  A mathematical model for pest management in Jatropha curcas with integrated pesticides - An optimal control approach , 2019, Ecological Complexity.

[31]  Xin Guo,et al.  Smooth Fit Principle for Impulse Control of Multidimensional Diffusion Processes , 2009, SIAM J. Control. Optim..

[32]  Hassan Dadashi.  Optimal investment strategy post retirement without ruin possibility: A numerical algorithm , 2020, J. Comput. Appl. Math..

[33]  H. Yoshioka,et al.  A singular stochastic control model for sustainable population management of the fish-eating waterfowl Phalacrocorax carbo. , 2018, Journal of environmental management.

[34]  Janice C. Eberly,et al.  Optimal Inattention to the Stock Market with Information Costs and Transactions Costs , 2009 .

[35]  Abner J. Salgado,et al.  Numerical analysis of strongly nonlinear PDEs * , 2016, Acta Numerica.

[36]  C. Sims.  Implications of rational inattention , 2003 .

[37]  H. Malchow,et al.  Wave pinning in competition-diffusion models in variable environments. , 2019, Journal of theoretical biology.

[38]  Eli P. Fenichel,et al.  Bioeconomic analysis supports the endangered species act , 2015, Journal of mathematical biology.

[39]  C. V. Hansen,et al.  An Adaptive E-Scheme for Conservation Laws , 2017, Lecture Notes in Computational Science and Engineering.

[40]  Avner Friedman,et al.  Optimal Stopping Problems in Stochastic Control , 1979 .

[41]  Naira Hovakimyan,et al.  Some Sufficient Conditions for Multi-Player Pursuit-Evasion Games with Continuous and Discrete Observations , 2009 .

[42]  Erik Ekström,et al.  Sequential testing of a Wiener process with costly observations , 2018 .

[43]  Alain Bensoussan,et al.  Estimation and Control of Dynamical Systems , 2018 .

[44]  H. Yoshioka.  A simplified stochastic optimization model for logistic dynamics with control-dependent carrying capacity , 2019, Journal of biological dynamics.

[45]  Georgios P. Georgiadis,et al.  Deadlines and Infrequent Monitoring in the Dynamic Provision of Public Goods , 2017 .

[46]  G. M. Moremedi,et al.  A simple mathematical model for Ebola in Africa , 2015, Journal of biological dynamics.

[47]  Tatiana P. Chernogorova,et al.  Finite volume difference scheme for a degenerate parabolic equation in the zero-coupon bond pricing , 2011, Math. Comput. Model..

[48]  Ł. Stettner.  On Ergodic control problems associated with optimal maintenance and inspection , 1984 .

[49]  Huyen Pham,et al.  Feynman-Kac representation for Hamilton-Jacobi-Bellman IPDE , 2012, 1212.2000.

[50]  M. T. Hoang,et al.  Complete Global Stability of a Metapopulation Model and Its Dynamically Consistent Discrete Models , 2018, Qualitative Theory of Dynamical Systems.

[51]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[52]  Maurice Robin.  Optimal maintenance and inspection : An impulsive control approach , 1978 .

[53]  Miglena N. Koleva,et al.  Fast computational approach to the Delta Greek of non-linear Black-Scholes equations , 2017, J. Comput. Appl. Math..

[54]  Peng Sun,et al.  Inducing Environmental Disclosures: A Dynamic Mechanism Design Approach , 2016, Oper. Res..

[55]  D. Finnoff,et al.  When is a “wait and see” approach to invasive species justified? , 2013 .

[56]  Motoh Tsujimura,et al.  An impulse control of a geometric Brownian motion with quadratic costs , 2006, Eur. J. Oper. Res..

[57]  Alex L. Marten,et al.  An options based bioeconomic model for biological and chemical control of invasive species , 2011 .

[58]  G. Pess,et al.  Increasing hydrologic variability threatens depleted anadromous fish populations , 2015, Global change biology.

[59]  Christof Schütte,et al.  Markov Control Processes with Rare State Observation: Theory and Application to Treatment Scheduling in HIV-1 , 2014 .

[60]  R. Veit,et al.  Partial Differential Equations in Ecology: Spatial Interactions and Population Dynamics , 1994 .

[61]  Lubin G. Vulkov,et al.  A finite volume difference scheme for a model of settling particle dispersion from an elevated source in an open-channel flow , 2014, Comput. Math. Appl..

[62]  Horst R. Thieme,et al.  Mathematics in Population Biology , 2003 .

[63]  F. Álvarez,et al.  Monetary Shocks in Models with Inattentive Producers , 2014, The Review of economic studies.

[64]  T. Sargent,et al.  Robust Control and Model Uncertainty , 2001 .

[65]  Qingling Zhang,et al.  Dynamical analysis and optimal control in a hybrid stochastic double delayed bioeconomic system with impulsive contaminants emission and Lévy jumps , 2019, Appl. Math. Comput..

[66]  Sören Christensen,et al.  Utility maximisation in a factor model with constant and proportional transaction costs , 2019, Finance Stochastics.

[67]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[68]  Hidekazu Yoshioka,et al.  Robust stochastic control modeling of dam discharge to suppress overgrowth of downstream harmful algae , 2018 .

[69]  Ayşen Davraz,et al.  Evaluation of water quality using water quality index (WQI) method and GIS in Aksu River (SW-Turkey). , 2017, The Science of the total environment.

[70]  Eli P. Fenichel,et al.  Tinbergen and tipping points: Could some thresholds be policy-induced? , 2016 .

[71]  Qiang Zhang,et al.  Robust optimal proportional reinsurance and investment strategy for an insurer with defaultable risks and jumps , 2019, J. Comput. Appl. Math..

[72]  Z. Gu,et al.  Assessment of water quality and phytoplankton community of Limpenaeus vannamei pond in intertidal zone of Hangzhou Bay, China , 2018, Aquaculture Reports.

[73]  I. Martins,et al.  Assessing the effects of temperature and salinity oscillations on a key mesopredator fish from European coastal systems. , 2018, The Science of the total environment.

[74]  Vincenzo Capasso,et al.  An Introduction to Continuous-Time Stochastic Processes , 2004, Modeling and Simulation in Science, Engineering and Technology.

[75]  Alexandru Hening,et al.  Optimal sustainable harvesting of populations in random environments , 2018, Stochastic Processes and their Applications.

[76]  B. Øksendal.  Stochastic Differential Equations , 1985 .

[77]  J. Shogren,et al.  Bioeconomics of invasive species: using real options theory to integrate ecology, economics, and risk management , 2016, Food Security.

[78]  Roland Glowinski,et al.  Recent Developments in Numerical Methods for Fully Nonlinear Second Order Partial Differential Equations , 2013, SIAM Rev..

[79]  Radoslav L. Valkov.  Convergence of a finite volume element method for a generalized Black-Scholes equation transformed on finite interval , 2014, Numerical Algorithms.

[80]  Bernt Øksendal,et al.  Optimal Stochastic Impulse Control with Delayed Reaction , 2008 .

[81]  H. Yoshioka,et al.  Stochastic differential game for management of non-renewable fishery resource under model ambiguity , 2018, Journal of biological dynamics.

[82]  S. Winkelmann.  Markov Decision Processes with Information Costs , 2013 .

[83]  Shaomin Wu,et al.  A condition-based maintenance policy for degrading systems with age- and state-dependent operating cost , 2017, Eur. J. Oper. Res..

[84]  Richard S. Palais,et al.  A simple proof of the Banach contraction principle , 2007 .

[85]  Radoslav L. Valkov.  Fitted finite volume method for a generalized Black–Scholes equation transformed on finite interval , 2013, Numerical Algorithms.

[86]  Xin Guo,et al.  Impulse Control of Multidimensional Jump Diffusions , 2009, SIAM J. Control. Optim..

[87]  Shige Peng,et al.  Necessary and sufficient condition for comparison theorem of 1-dimensional stochastic differential equations , 2006 .

[88]  Anastasios Xepapadeas,et al.  Robust control of parabolic stochastic partial differential equations under model uncertainty , 2019, Eur. J. Control.

[89]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1991 .

[90]  Christof Schütte,et al.  Optimal Treatment Strategies in the Context of ‘Treatment for Prevention’ against HIV-1 in Resource-Poor Settings , 2015, PLoS computational biology.

[91]  B. Øksendal,et al.  Optimal harvesting from a population in a stochastic crowded environment. , 1997, Mathematical biosciences.

[92]  Dante Kalise,et al.  Optimal control : novel directions and applications , 2017 .