Ergodic Control of Diffusion Processes

Results concerning existence and characterization of optimal controls for ergodic control of nondegenerate diffusion processes are described. Extensions to the general ‘controlled martingale problem’ are indicated, which cover in particular degenerate diffusions and some infinite dimensional problems. In conclusion, some related problems and open issues are discussed. Mathematics Subject Classification (2000). Primary 93E20; Secondary 60H20.

[1]  W. Feller.  Non-Markovian Processes with the Semigroup Property , 1959 .

[2]  S. Agmon,et al.  Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions. I , 1959 .

[3]  R. Khas'minskii.  Ergodic Properties of Recurrent Diffusion Processes and Stabilization of the Solution to the Cauchy Problem for Parabolic Equations , 1960 .

[4]  L. Dubins.  On extreme points of convex sets , 1962 .

[5]  M. Freidlin.  Diffusion Processes with Reflection and Problems with a Directional Derivative on a Manifold with a Boundary , 1963 .

[6]  D. Freedman,et al.  Measurable sets of measures. , 1964 .

[7]  Y. Kogan.  On Optimal Control of a Non-Terminating Diffusion Process with Reflection , 1969 .

[8]  V. Benes.  Existence of Optimal Strategies Based on Specified Information, for a Class of Stochastic Decision Problems , 1970 .

[9]  S. Ross.  Average cost semi-markov decision processes , 1970, Journal of Applied Probability.

[10]  R. Rishel.  Necessary and Sufficient Dynamic Programming Conditions for Continuous Time Stochastic Optimal Control , 1970 .

[11]  E. Wong.  Representation of Martingales, Quadratic Variation and Applications , 1971 .

[12]  S. Varadhan,et al.  Diffusion processes with boundary conditions , 1971 .

[13]  J. Willems.  Dissipative dynamical systems part I: General theory , 1972 .

[14]  A. Zvonkin.  A TRANSFORMATION OF THE PHASE SPACE OF A DIFFUSION PROCESS THAT REMOVES THE DRIFT , 1974 .

[15]  Daniel H. Wagner.  Survey of Measurable Selection Theorems , 1977 .

[16]  D. W. Stroock,et al.  Multidimensional Diffusion Processes , 1979 .

[17]  N. Krylov.  Controlled Diffusion Processes , 1980 .

[18]  W. Fleming.  Measure-valued processes in the control of partially-observable stochastic systems , 1980 .

[19]  S. Mitter,et al.  New results on the innovations problem for non-linear filtering , 1981 .

[20]  C. Kenig,et al.  Examples of singular parabolic measures and singular transition probability densities , 1981 .

[21]  M. Robin.  On Some Impulse Control Problems with Long Run Average Cost , 1981 .

[22]  W. Fleming,et al.  Optimal Control for Partially Observed Diffusions , 1982 .

[23]  P. Lions.  Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness , 1983 .

[24]  P. L. Linos.  Optimal control of diffustion processes and hamilton-jacobi-bellman equations part I: the dynamic programming principle and application , 1983 .

[25]  M. Robin.  Long-term average cost control problems for continuous time Markov processes: A survey , 1983 .

[26]  V. Borkar,et al.  Ergodic control problem for one-dimensional diffusions with near-monotone cost , 1984 .

[27]  P. Lions,et al.  Stochastic differential equations with reflecting boundary conditions , 1984 .

[28]  C. Striebel.  Martingale conditions for the optimal control of continuous time stochastic systems , 1984 .

[29]  Ulrich G. Haussmann.  L'equation de Zakai et le problème séparé du contrôle optimal stochastique , 1985 .

[30]  V. Borkar,et al.  Corrections to :20Ergodic control problem for one-dimensional diffusions with near-monotone cost” , 1986 .

[31]  A. Veretennikov.  On Strong Solutions of Itô Stochastic Equations with Jumps , 1988 .

[32]  M. K. Ghosh,et al.  Ergodic control of multidimensional diffusions 1: the existence results , 1988 .

[33]  V. Borkar.  A topology for Markov controls , 1989 .

[34]  Richard H. Stockbridg.  Time-average control of martingale problems: the hamilton-jacobi-bellman equation , 1989 .

[35]  J. Menaldi,et al.  Ergodic control of reflected diffusions with jumps , 1989 .

[36]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[37]  M. K. Ghosh,et al.  Ergodic control of multidimensional diffusions, II: Adaptive control , 1990 .

[38]  R. Stockbridge.  Time-Average Control of Martingale Problems: A Linear Programming Formulation , 1990 .

[39]  R. Stockbridge.  Time-Average Control of Martingale Problems: Existence of a Stationary Solution , 1990 .

[40]  M. K. Ghosh,et al.  Controlled diffusions with constraints , 1990 .

[41]  B. Perthame,et al.  Ergodic problem for optimal stochastic switching , 1990 .

[42]  V. Borkar.  On extremal solutions to stochastic control problems , 1991 .

[43]  Michael I. Taksar,et al.  Singular ergodic control for multidimensional Gaussian processes , 1992, Math. Control. Signals Syst..

[44]  M. K. Ghosh,et al.  Stochastic differential games: Occupation measure based approach , 1992 .

[45]  A. Bensoussan,et al.  On Bellman equations of ergodic control in n. , 1992 .

[46]  A. Bensoussan,et al.  On Bellman equations of ergodic control in Rn , 1992 .

[47]  M. K. Ghosh,et al.  Optimal control of switching diffusions with application to flexible manufacturing systems , 1993 .

[48]  R. Karandikar,et al.  Invariant Measures and Evolution Equations for Markov Processes Characterized Via Martingale Problems , 1993 .

[49]  M. K. Ghosh,et al.  Discrete-time controlled Markov processes with average cost criterion: a survey , 1993 .

[50]  V. Borkar,et al.  On Extremal Solutions of Controlled Nonlinear Filtering Equations , 1995 .

[51]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[52]  I. Gyöngy,et al.  Existence of strong solutions for Itô's stochastic equations via approximations , 1996, Stochastics and Partial Differential Equations: Analysis and Computations.

[53]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[54]  Gopal K. Basak,et al.  Ergodic control of degenerate diffusions , 1997 .

[55]  M. Arisawa.  Ergodic problem for the Hamilton-Jacobi-Bellman equation. I. Existence of the ergodic attractor , 1997 .

[56]  M. K. Ghosh,et al.  Ergodic Control of Switching Diffusions , 1997 .

[57]  J. Menaldi,et al.  Ergodic control of reflected diffusions with jumps , 1997 .

[58]  P. Lions,et al.  ON ERGODIC STOCHASTIC CONTROL , 1998 .

[59]  T. Kurtz,et al.  Existence of Markov Controls and Characterization of Optimal Markov Controls , 1998 .

[60]  M. Arisawa.  Ergodic problem for the Hamilton-Jacobi-Bellman equation. II , 1998 .

[61]  M. Safonov.  Nonuniqueness for second-order elliptic equations with measurable coefficients , 1999 .

[62]  M. K. Ghosh,et al.  Harnack's inequality for cooperative weakly coupled elliptic systems , 1999 .

[63]  Numerical Comparison of Controls and Verification of Optimality for Stochastic Control Problems , 2000 .

[64]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[65]  T. Kurtz,et al.  Stationary Solutions and Forward Equations for Controlled and Singular Martingale Problems , 2001 .

[66]  V. Bogachev,et al.  ON REGULARITY OF TRANSITION PROBABILITIES AND INVARIANT MEASURES OF SINGULAR DIFFUSIONS UNDER MINIMAL CONDITIONS , 2001 .

[67]  V. Borkar.  Dynamic programming for ergodic control with partial observations , 2003 .

[68]  V. Borkar,et al.  A Note on Stochastic Dissipativeness , 2003 .

[69]  B. Sirakov,et al.  Harnack type estimates for nonlinear elliptic systems and applications , 2004 .

[70]  Vivek S. Borkar,et al.  Ergodic Control for Constrained Diffusions: Characterization Using HJB Equations , 2004, SIAM J. Control. Optim..

[71]  V. Borkar,et al.  A further remark on dynamic programming for partially observed Markov processes , 2004 .

[72]  Wei Wu,et al.  Optimal power allocation for a time-varying wireless channel under heavy-traffic approximation , 2006, IEEE Transactions on Automatic Control.

[73]  Vivek S. Borkar,et al.  Singular Perturbations in Ergodic Control of Diffusions , 2007, SIAM J. Control. Optim..

[74]  Vivek S. Borkar,et al.  Uniform Recurrence Properties of Controlled Diffusions and Applications to Optimal Control , 2010, SIAM J. Control. Optim..