Controlled equilibrium selection in stochastically perturbed dynamics

We consider a dynamical system with finitely many equilibria and perturbed by small noise, in addition to being controlled by an `expensive' control. The controlled process is optimal for an ergodic criterion with a running cost that consists the sum of the control effort and a penalty function on the state space. We study the optimal stationary distribution of the controlled process as the variance of the noise becomes vanishingly small. It is shown that depending on the relative magnitudes of the noise variance and the `running cost' for control, one can identify three regimes, in each of which the optimal control forces the invariant distribution of the process to concentrate near equilibria that can be characterized according to the regime. We also obtain moment bounds for the optimal stationary distribution. Moreover, we show that in the vicinity of the points of concentration the density of optimal stationary distribution approximates the density of a Gaussian, and we explicitly solve for its covariance matrix.

[1]  S. Smale.  On Gradient Dynamical Systems , 1961 .

[2]  K. Meyer.  ENERGY FUNCTIONS FOR MORSE SMALE SYSTEMS. , 1968 .

[3]  J. Willems.  Least squares stationary optimal control and the algebraic Riccati equation , 1971 .

[4]  K. Mårtensson,et al.  On the matrix riccati equation , 1971, Inf. Sci..

[5]  K. Mrtensson.  On the matrix riccati equation , 1971 .

[6]  V. Kučera.  A contribution to matrix quadratic equations , 1972 .

[7]  Zeev Schuss,et al.  Theory and Applications of Stochastic Differential Equations , 1980 .

[8]  G. Parisi,et al.  A Theory of Stochastic Resonance in Climatic Change , 1983 .

[9]  S. Sheu.  Asymptotic behavior of the invariant density of a diffusion Markov process with small diffusion , 1986 .

[10]  M. V. Day,et al.  Recent progress on the small parameter exit problem , 1987 .

[11]  Ben-Zion Bobrovsky,et al.  Error bounds for the nonlinear filtering of signals with small diffusion coefficients , 1988, IEEE Trans. Inf. Theory.

[12]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[13]  J. Filar,et al.  Some comments on a theorem of Hardy and Littlewood , 1992 .

[14]  L. Arnold.  Stochastic Differential Equations: Theory and Applications , 1992 .

[15]  A. Bensoussan,et al.  On Bellman equations of ergodic control in n. , 1992 .

[16]  A. Bensoussan,et al.  On Bellman equations of ergodic control in Rn , 1992 .

[17]  On the optimal tracking problem , 1992 .

[18]  Frank Moss,et al.  5. Stochastic Resonance: From the Ice Ages to the Monkey's Ear , 1994 .

[19]  V. Borkar.  Probability Theory: An Advanced Course , 1995 .

[20]  I. Gyöngy,et al.  Existence of strong solutions for Itô's stochastic equations via approximations , 1996, Stochastics and Partial Differential Equations: Analysis and Computations.

[21]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[22]  T. Kurtz,et al.  Existence of Markov Controls and Characterization of Optimal Markov Controls , 1998 .

[23]  Alain Bensoussan,et al.  Regularity Results for Nonlinear Elliptic Systems and Applications , 2002 .

[24]  Barbara Gentz,et al.  Metastability in simple climate models: Pathwise analysis of slowly driven Langevin equations , 2002 .

[25]  Michael Röckner,et al.  Large deviations for invariant measures of stochastic reaction–diffusion systems with multiplicative noise and non-Lipschitz reaction term , 2005 .

[26]  Michael Röckner,et al.  Strong solutions of stochastic equations with singular time dependent drift , 2005 .

[27]  A. Rhandi,et al.  Global properties of invariant measures , 2005 .

[28]  E. Olivieri,et al.  Large deviations and metastability , 2005 .

[29]  N. Berglund,et al.  Noise-Induced Phenomena in Slow-Fast Dynamical Systems: A Sample-Paths Approach , 2005 .

[30]  Zhong-Ping Jiang,et al.  Enhancement of Stochastic Resonance Using Optimization Theory , 2006, Commun. Inf. Syst..

[31]  K. Wiesenfeld,et al.  Potential energy landscape and finite-state models of array-enhanced stochastic resonance. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[32]  Gregoire Nicolis,et al.  Stochastic resonance , 2007, Scholarpedia.

[33]  Zhong-Ping Jiang,et al.  An investigation of two-dimensional parameter-induced stochastic resonance and applications in nonlinear image processing , 2009 .

[34]  V. Borkar,et al.  Small noise asymptotics for invariant densities for a class of diffusions: A control theoretic view , 2009, 1107.2277.

[35]  Jin Feng,et al.  Short-Maturity Asymptotics for a Fast Mean-Reverting Heston Stochastic Volatility Model , 2010, SIAM J. Financial Math..

[36]  Daniel W. Repperger,et al.  Stochastic resonance–a nonlinear control theory interpretation , 2010, Int. J. Syst. Sci..

[37]  Naoyuki Ichihara,et al.  Recurrence and Transience of Optimal Feedback Processes Associated with Bellman Equations of Ergodic Type , 2011, SIAM J. Control. Optim..

[38]  Naoyuki Ichihara.  Large time asymptotic problems for optimal stochastic control with superlinear cost , 2012 .

[39]  On positive and probability solutions to the stationary Fokker-Planck-Kolmogorov equation , 2012 .

[40]  Naoyuki Ichihara,et al.  Large Time Behavior of Solutions of Hamilton-Jacobi-Bellman Equations with Quadratic Nonlinearity in Gradients , 2013, SIAM J. Math. Anal..

[41]  R. Brockett.  Finite Dimensional Linear Systems , 2015 .

[42]  Naoyuki Ichihara.  The generalized principal eigenvalue for Hamilton–Jacobi–Bellman equations of ergodic type , 2015 .

[43]  SUPPLEMENT TO “ CONTROLLED EQUILIBRIUM SELECTION IN STOCHASTICALLY PERTURBED DYNAMICS , 2017 .

[44]  Abubakr Gafar Abdalla,et al.  Probability Theory , 2017, Encyclopedia of GIS.