Controlled diffusion processes

This article gives an overview of the developments in controlled diffusion processes, emphasizing key results regarding existence of optimal controls and their characterization via dynamic programming for a variety of cost criteria and structural assumptions. Stochastic maximum principle and control under partial observations (equivalently, control of nonlinear filters) are also discussed. Several other related topics are briefly sketched.

[1]  Harold J. Kushner,et al.  A maximum principle for stochastic control systems , 1964 .

[2]  O. Ladyženskaja.  Linear and Quasilinear Equations of Parabolic Type , 1968 .

[3]  V. Benes.  Existence of Optimal Strategies Based on Specified Information, for a Class of Stochastic Decision Problems , 1970 .

[4]  R. Rishel.  Necessary and Sufficient Dynamic Programming Conditions for Continuous Time Stochastic Optimal Control , 1970 .

[5]  P. Varaiya,et al.  On the Solutions of a Stochastic Control System. II , 1971 .

[6]  V. Benes.  Existence of Optimal Stochastic Control Laws , 1971 .

[7]  L. Young,et al.  Lectures on the Calculus of Variations and Optimal Control Theory. , 1971 .

[8]  T. Parthasarathy.  Selection Theorems and Their Applications , 1972 .

[9]  P. Varaiya,et al.  Dynamic Programming Conditions for Partially Observable Stochastic Systems , 1973 .

[10]  Harold J. Kushner,et al.  Existence results for optimal stochastic controls , 1975 .

[11]  J. Bismut,et al.  Théorie probabiliste du contrôle des diffusions , 1976 .

[12]  P. Varaiya.  N-player stochastic differential games , 1976 .

[13]  D. Gilbarg,et al.  Elliptic Partial Differential Equa-tions of Second Order , 1977 .

[14]  P. Kloeden,et al.  Numerical Solution of Stochastic Differential Equations , 1992 .

[15]  I. Gihman,et al.  Controlled Stochastic Processes , 1979 .

[16]  D. W. Stroock,et al.  Multidimensional Diffusion Processes , 1979 .

[17]  L. Young.  Lectures on the Calculus of Variations and Optimal Control Theory , 1980 .

[18]  N. Krylov.  Controlled Diffusion Processes , 1980 .

[19]  Norbert Christopeit.  Existence of optimal stochastic controls under partial observation , 1980 .

[20]  Alain Bensoussan,et al.  Stochastic Production Planning with Production Constraints , 1980 .

[21]  Albert N. Shiryaev,et al.  Optimal Stopping Rules , 1980, International Encyclopedia of Statistical Science.

[22]  S. Mitter,et al.  New results on the innovations problem for non-linear filtering , 1981 .

[23]  A. Veretennikov.  ON STRONG SOLUTIONS AND EXPLICIT FORMULAS FOR SOLUTIONS OF STOCHASTIC INTEGRAL EQUATIONS , 1981 .

[24]  C. Kenig,et al.  Examples of singular parabolic measures and singular transition probability densities , 1981 .

[25]  Frank Spitzer.  Optimal Stopping Rules (A. N. Shiryayev) , 1981 .

[26]  A. Bensoussan.  Stochastic control by functional analysis methods , 1982 .

[27]  Suzanne Lenhart,et al.  Integro-differential operators associated with diffusion processes with jumps , 1982 .

[28]  Wendell H. Fleming,et al.  Nonlinear Semigroup for Controlled Partially Observed Diffusions. , 1982 .

[29]  Alain Bensoussan,et al.  Applications of Variational Inequalities in Stochastic Control , 1982 .

[30]  P. Echeverría.  A criterion for invariant measures of markov processes , 1982 .

[31]  Alain Bensoussan.  Optimal control of partially observed diffusions , 1982 .

[32]  Alain Bensoussan,et al.  Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions , 1983 .

[33]  V. Borkar.  Existence of optimal controls for partially observed diffusions , 1983 .

[34]  P. Lions.  Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness , 1983 .

[35]  P. L. Linos.  Optimal control of diffustion processes and hamilton-jacobi-bellman equations part I: the dynamic programming principle and application , 1983 .

[36]  Łukasz Stettner,et al.  On impulsive control with long run average cost criterion , 1983 .

[37]  Alain Bensoussan,et al.  Impulse Control and Quasi-Variational Inequalities , 1984 .

[38]  A. Shiryayev,et al.  Statistics of Random Processes I: General Theory , 1984 .

[39]  C. Striebel.  Martingale conditions for the optimal control of continuous time stochastic systems , 1984 .

[40]  Ulrich G. Haussmann.  L'equation de Zakai et le problème séparé du contrôle optimal stochastique , 1985 .

[41]  J. Harrison,et al.  Brownian motion and stochastic flow systems , 1986 .

[42]  U. Haussmann.  Existence of optimal markovian controls for degenerate diffusions , 1986 .

[43]  Andrew J. Heunis.  On the Prevalence of Stochastic Differential Equations with Unique Strong Solutions , 1986 .

[44]  U. Haussmann,et al.  A stochastic maximum principle for optimal control of diffusions , 1986 .

[45]  V. Borkar.  A remark on the attainable distributions of controlled diffusions , 1986 .

[46]  KarouiNicole El,et al.  Compactification methods in the control of degenerate diffusions: existence of an optimal control , 1987 .

[47]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[48]  V. Borkar.  Control of a partially observed diffusion up to an exit time , 1987 .

[49]  Bohdan Maslowski,et al.  A stochastic maximum principle for optimal control of diffusions , 1988, Acta Applicandae Mathematicae.

[50]  S. Shreve.  An Introduction to Singular Stochastic Control , 1988 .

[51]  P. Caines.  Linear Stochastic Systems , 1988 .

[52]  M. K. Ghosh,et al.  Ergodic control of multidimensional diffusions 1: the existence results , 1988 .

[53]  B. Perthame,et al.  An Impulsive Control Problem with State Constraint , 1989 .

[54]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[55]  Xun Yu Zhouf.  The connection between the maximum principle and dynamic programming in stochastic control , 1990 .

[56]  M. K. Ghosh,et al.  Ergodic control of multidimensional diffusions, II: Adaptive control , 1990 .

[57]  R. Stockbridge.  Time-Average Control of Martingale Problems: Existence of a Stationary Solution , 1990 .

[58]  H. Kushner.  Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems , 1990 .

[59]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1990, 29th IEEE Conference on Decision and Control.

[60]  M. K. Ghosh,et al.  Controlled diffusions with constraints , 1990 .

[61]  B. Perthame,et al.  Ergodic problem for optimal stochastic switching , 1990 .

[62]  Vivek S. Borkar,et al.  Self-tuning control of diffusions without the identifiability condition , 1991 .

[63]  On extremal solutions to stochastic control problems, II , 1991 .

[64]  V. Borkar.  On extremal solutions to stochastic control problems , 1991 .

[65]  M. K. Ghosh,et al.  Stochastic differential games: Occupation measure based approach , 1992 .

[66]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[67]  A. Bensoussan,et al.  On Bellman equations of ergodic control in Rn , 1992 .

[68]  J. Yong,et al.  Finite horizon stochastic optimal switching and impulse controls with a viscosity solution approach , 1993 .

[69]  M. K. Ghosh,et al.  Optimal control of switching diffusions with application to flexible manufacturing systems , 1993 .

[70]  B. Pasik-Duncan,et al.  Almost self-optimizing strategies for the adaptive control of diffusion processes , 1994 .

[71]  J. Quadrat.  Numerical methods for stochastic control problems in continuous time , 1994 .

[72]  H. Nagai.  Bellman equations of risk sensitive control , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[73]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[74]  Ioannis Karatzas,et al.  Lectures on the Mathematics of Finance , 1996 .

[75]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[76]  V. Borkar,et al.  Occupation measures for controlled Markov processes: characterization and optimality , 1996 .

[77]  N. Karoui,et al.  Backward Stochastic Differential Equations , 1997 .

[78]  N. Nadirashvili.  Nonuniqueness in the martingale problem and the Dirichlet problem for uniformly elliptic operators , 1997 .

[79]  Jan A. Van Mieghem,et al.  Dynamic Control of Brownian Networks: State Space Collapse and Equivalent Workload Formulations , 1997 .

[80]  Financial mathematics : lectures given at the 3rd session of the Centro internazionale matematico estivo (C.I.M.E.) held in Bressanone, Italy, July 8-13, 1996 , 1997 .

[81]  P. Bassanini,et al.  Elliptic Partial Differential Equations of Second Order , 1997 .

[82]  R. Stockbridge,et al.  Approximation of Infinite-Dimensional Linear Programming Problems which Arise in Stochastic Control , 1998 .

[83]  P. Dupuis,et al.  Robust Properties of Risk-Sensitive Control , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[84]  Shanjian Tang,et al.  The Maximum Principle for Partially Observed Optimal Control of Stochastic Differential Equations , 1998 .

[85]  J. Yong,et al.  Dynamic programming for multidimensional stochastic control problems , 1999 .

[86]  C. Charalambous,et al.  Necessary Conditions of Optimization for Partially Observed Controlled Diffusions , 1999 .

[87]  M. K. Ghosh,et al.  Optimal Control of a Stochastic Hybrid System with Discounted Cost , 1999 .

[88]  Markov marginal problems and their applications to Markov optimal control , 1999 .

[89]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[90]  Numerical Comparison of Controls and Verification of Optimality for Stochastic Control Problems , 2000 .

[91]  Lukasz Kruk.  Optimal Policies for n-Dimensional Singular Stochastic Control Problems. Part II: The Radially Symmetric Case. Ergodic Control , 2000, SIAM J. Control. Optim..

[92]  Lukasz Kruk.  Optimal Policies for n-Dimensional Singular Stochastic Control Problems Part I: The Skorokhod Problem , 2000, SIAM J. Control. Optim..

[93]  A. Bensoussan,et al.  Stochastic Games for N Players , 2000 .

[94]  Tomasz R. Bielecki,et al.  Risk sensitive asset management with transaction costs , 2000, Finance Stochastics.

[95]  W. Fleming,et al.  Risk‐Sensitive Control and an Optimal Investment Model , 2000 .

[96]  X. Zhou,et al.  Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework , 2000 .

[97]  Fausto Gozzi,et al.  Hamilton–Jacobi–Bellman Equations for the Optimal Control of the Duncan–Mortensen–Zakai Equation☆☆☆ , 2000 .

[98]  Luis H. R. Alvarez,et al.  Singular Stochastic Control, Linear Diffusions, and Optimal Stopping: A Class of Solvable Problems , 2000, SIAM J. Control. Optim..

[99]  L. Kruk.  Optimal policies for some n-dimensional singular stochastic control problems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[100]  H. Nagai,et al.  Risk-sensitive portfolio optimization on infinite time horizon , 2002 .

[101]  Michael I. Taksar,et al.  Dynkin Games via Dirichlet Forms and Singular Control of One-Dimensional Diffusions , 2002, SIAM J. Control. Optim..

[102]  Richard H. Stockbridge,et al.  Linear Programming Formulation for Optimal Stopping Problems , 2001, SIAM J. Control. Optim..

[103]  Ralf Korn,et al.  A Stochastic Control Approach to Portfolio Problems with Stochastic Interest Rates , 2001, SIAM J. Control. Optim..

[104]  Y. Kabanov,et al.  Two-scale stochastic systems , 2002 .

[105]  Bernard Lapeyre,et al.  Introduction to Monte-Carlo Methods for Transport and Diffusion Equations , 2003 .

[106]  Paul Glasserman,et al.  Monte Carlo Methods in Financial Engineering , 2003 .

[107]  A probabilistic approach to second order variational inequalities with bilateral constraints , 2003, math/0406076.

[108]  P. Ma.  Forward Backward Stochastic Differential Equations , 2003 .

[109]  Amarjit Budhiraja.  An Ergodic Control Problem for Constrained Diffusion Processes: Existence of Optimal Markov Control , 2003, SIAM J. Control. Optim..

[110]  Boris M. Miller,et al.  Singular Stochastic Control Problems , 2004, SIAM J. Control. Optim..

[111]  J. Michael Harrison,et al.  Dynamic Scheduling of a Multiclass Queue in the Halfin-Whitt Heavy Traffic Regime , 2004, Oper. Res..

[112]  N. Krylov,et al.  On weak uniqueness for some diffusions with discontinuous coefficients , 2004 .

[113]  Luis H. R. Alvarez,et al.  Stochastic Forest Stand Value and Optimal Timber Harvesting , 2003, SIAM J. Control. Optim..

[114]  Claudia Klüppelberg,et al.  A geometric approach to portfolio optimization in models with transaction costs , 2004, Finance Stochastics.

[115]  Harold J. Kushner.  Numerical Approximations for Stochastic Differential Games: The Ergodic Case , 2004, SIAM J. Control. Optim..

[116]  H. Pham,et al.  Optimal Partially Reversible Investment with Entry Decision and General Production Function , 2005 .

[117]  H. Pham.  On the Smooth-Fit Property for One-Dimensional Optimal Switching Problem , 2004, math/0410285.