On some recent aspects of stochastic control and their applications

This paper is a survey on some recent aspects and developments in stochastic control. We discuss the two main historical approaches, Bellman's optimality principle and Pontryagin's maximum principle, and their modern exposition with viscosity solutions and backward stochastic differential equations. Some original proofs are presented in a unifying context including degenerate singular control problems. We emphasize key results on characterization of optimal control for diffusion processes, with a view towards applications. Some examples in finance are detailed with their explicit solutions. We also discuss numerical issues and open questions.

[1]  R. Bellman.  Dynamic programming. , 1957, Science.

[2]  Wendell H. Fleming,et al.  Optimal Control of Partially Observable Diffusions , 1968 .

[3]  O. Ladyženskaja.  Linear and Quasilinear Equations of Parabolic Type , 1968 .

[4]  P. Varaiya,et al.  Dynamic Programming Conditions for Partially Observable Stochastic Systems , 1973 .

[5]  K. Ioannis.  On a stochastic representation for the principal eigenvalue of a second-order differential equation , 1980 .

[6]  N. Karoui.  Les Aspects Probabilistes Du Controle Stochastique , 1981 .

[7]  西尾 真喜子.  Lectures on stochastic control theory , 1981 .

[8]  P. Lions.  Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness , 1983 .

[9]  W. Grassman.  Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory (Harold J. Kushner) , 1986 .

[10]  P. Lions.  Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. Part I: the case of bounded stochastic evolutions , 1988 .

[11]  T. Zariphopoulou.  Optimal Investment - Consumption Models With Constraints , 1989 .

[12]  R. Elliott,et al.  The partially observed stochastic minimum principle , 1989 .

[13]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[14]  S. Peng.  A general stochastic maximum principle for optimal control problems , 1990 .

[15]  Marianne Akian.  Analyse de l’algorithme multigrille FMGH de résolution d’équations d’Hamilton-Jacobi-Bellman , 1990 .

[16]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1990, 29th IEEE Conference on Decision and Control.

[17]  S. Peng,et al.  Adapted solution of a backward stochastic differential equation , 1990 .

[18]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1991 .

[19]  Ben G. Fitzpatrick,et al.  Numerical Methods for an Optimal Investment-Consumption Model , 1991, Math. Oper. Res..

[20]  A. Bensoussan,et al.  An ergodic control problem arising from the principal eigenfunction of an elliptic operator , 1991 .

[21]  A. Bensoussan.  Stochastic Control of Partially Observable Systems , 1992 .

[22]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[23]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[24]  J. Yong,et al.  Finite horizon stochastic optimal switching and impulse controls with a viscosity solution approach , 1993 .

[25]  F. Antonelli,et al.  Backward-Forward Stochastic Differential Equations , 1993 .

[26]  X. Zhou.  On the necessary conditions of optimal controls for stochastic partial differential equations , 1993 .

[27]  A. Tourin,et al.  Numerical schemes for investment models with singular transactions , 1994 .

[28]  B. Øksendal,et al.  Optimal Switching in an Economic Activity Under Uncertainty , 1994 .

[29]  J. Yong,et al.  Solving forward-backward stochastic differential equations explicitly — a four step scheme , 1994 .

[30]  H. Soner,et al.  Optimal Investment and Consumption with Transaction Costs , 1994 .

[31]  J. Quadrat.  Numerical methods for stochastic control problems in continuous time , 1994 .

[32]  A. Shiryaev,et al.  Optimization of the flow of dividends , 1995 .

[33]  S. Peng,et al.  Solution of forward-backward stochastic differential equations , 1995 .

[34]  Jin Ma.  Forward-backward stochastic differential equations and their applications in finance , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[35]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[36]  N. Karoui,et al.  Dynamic Programming and Pricing of Contingent Claims in an Incomplete Market , 1995 .

[37]  H. Soner,et al.  Optimal Replication of Contingent Claims Under Portfolio Constraints , 1996 .

[38]  J. Douglas,et al.  Numerical methods for forward-backward stochastic differential equations , 1996 .

[39]  S. Peng,et al.  Backward Stochastic Differential Equations in Finance , 1997 .

[40]  N. Karoui,et al.  Backward Stochastic Differential Equations , 1997 .

[41]  Ralf Korn,et al.  Portfolio optimisation with strictly positive transaction costs and impulse control , 1998, Finance Stochastics.

[42]  S. Pliska,et al.  Risk-Sensitive Dynamic Asset Management , 1999 .

[43]  É. Pardoux,et al.  Forward-backward stochastic differential equations and quasilinear parabolic PDEs , 1999 .

[44]  M. Schweizer.  A guided tour through quadratic hedging approaches , 1999 .

[45]  Jakša Cvitanić,et al.  Super-replication in stochastic volatility models under portfolio constraints , 1999, Journal of Applied Probability.

[46]  Philippe Artzner,et al.  Coherent Measures of Risk , 1999 .

[47]  Huyên Pham,et al.  A closed-form solution to the problem of super-replication under transaction costs , 1999, Finance Stochastics.

[48]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[49]  Nizar Touzi,et al.  Superreplication Under Gamma Constraints , 2000, SIAM J. Control. Optim..

[50]  Huyên Pham,et al.  On quadratic hedging in continuous time , 2000, Math. Methods Oper. Res..

[51]  Xun Yu Zhou,et al.  Relationship Between Backward Stochastic Differential Equations and Stochastic Controls: A Linear-Quadratic Approach , 2000, SIAM J. Control. Optim..

[52]  N. Krylov.  On the rate of convergence of finite-difference approximations for Bellmans equations with variable coefficients , 2000 .

[53]  M. Kobylanski.  Backward stochastic differential equations and partial differential equations with quadratic growth , 2000 .

[54]  W. Fleming,et al.  Risk‐Sensitive Control and an Optimal Investment Model , 2000 .

[55]  M. Zervos,et al.  A model for investment decisions with switching costs , 2001 .

[56]  Xin Guo.  An explicit solution to an optimal stopping problem with regime switching , 2001, Journal of Applied Probability.

[57]  Thaleia Zariphopoulou,et al.  A solution approach to valuation with unhedgeable risks , 2001, Finance Stochastics.

[58]  Nizar Touzi,et al.  Stochastic Target Problems, Dynamic Programming, and Viscosity Solutions , 2002, SIAM J. Control. Optim..

[59]  Shanjian Tang,et al.  Global Adapted Solution of One-Dimensional Backward Stochastic Riccati Equations, with Application to the Mean-Variance Hedging , 2002 .

[60]  H. Pham,et al.  Smooth Solutions to Optimal Investment Models with Stochastic Volatilities and Portfolio Constraints , 2002 .

[61]  F. Delarue.  On the existence and uniqueness of solutions to FBSDEs in a non-degenerate case , 2002 .

[62]  P. Protter,et al.  Numberical Method for Backward Stochastic Differential Equations , 2002 .

[63]  Bernt Øksendal,et al.  Optimal Consumption and Portfolio with Both Fixed and Proportional Transaction Costs , 2001, SIAM J. Control. Optim..

[64]  Daniel Hernández-Hernández,et al.  An optimal consumption model with stochastic volatility , 2003, Finance Stochastics.

[65]  M. Mania,et al.  BACKWARD STOCHASTIC PDE AND IMPERFECT HEDGING , 2003 .

[66]  Huyên Pham,et al.  A risk-sensitive control dual approach to a large deviations control problem , 2003, Syst. Control. Lett..

[67]  G. Pagès,et al.  Error analysis of the optimal quantization algorithm for obstacle problems , 2003 .

[68]  Monique Jeanblanc,et al.  Optimal Investment and Consumption Decisions when Time-Horizon is Uncertain , 2003 .

[69]  Huyên Pham,et al.  A large deviations approach to optimal long term investment , 2003, Finance Stochastics.

[70]  G. Pagès,et al.  Optimal quantization methods and applications to numerical problems in finance , 2004 .

[71]  Martin Schweizer,et al.  Mean-variance hedging and stochastic control: beyond the Brownian setting , 2004, IEEE Transactions on Automatic Control.

[72]  Philip Protter,et al.  Liquidity Risk and Arbitrage Pricing Theory , 2004 .

[73]  Claudia Klüppelberg,et al.  A geometric approach to portfolio optimization in models with transaction costs , 2004, Finance Stochastics.

[74]  Philip Protter,et al.  Noname manuscript No. (will be inserted by the editor) Liquidity Risk and Arbitrage Pricing Theory , 2003 .

[75]  Huyên Pham,et al.  Wealth-path dependent utility maximization in incomplete markets , 2004, Finance Stochastics.

[76]  G. Pagès,et al.  AN OPTIMAL MARKOVIAN QUANTIZATION ALGORITHM FOR MULTI-DIMENSIONAL STOCHASTIC CONTROL PROBLEMS , 2004 .

[77]  B. Bouchard,et al.  Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations , 2004 .

[78]  Gordan Zitkovic.  Utility Maximization with a Stochastic Clock and an Unbounded Random Endowment , 2005, math/0503516.

[79]  Anne Gundel,et al.  Robust utility maximization for complete and incomplete market models , 2005, Finance Stochastics.

[80]  H. Pham,et al.  Optimal Partially Reversible Investment with Entry Decision and General Production Function , 2005 .

[81]  E. Gobet,et al.  A regression-based Monte Carlo method to solve backward stochastic differential equations , 2005, math/0508491.

[82]  V. Borkar.  Controlled diffusion processes , 2005, math/0511077.

[83]  Alexander Schied,et al.  Optimal Investments for Robust Utility Functionals in Complete Market Models , 2005, Math. Oper. Res..

[84]  Guy Barles,et al.  Error Bounds for Monotone Approximation Schemes for Hamilton-Jacobi-Bellman Equations , 2005, SIAM J. Numer. Anal..

[85]  Ying Hu,et al.  On a Class of Stochastic Optimal Control Problems Related to BSDEs with Quadratic Growth , 2006, SIAM J. Control. Optim..

[86]  S. Menozzi,et al.  A Forward-Backward Stochastic Algorithm For Quasi-Linear PDEs , 2006, math/0603250.

[87]  Huyên Pham,et al.  A model of optimal portfolio selection under liquidity risk and price impact , 2006, Finance Stochastics.

[88]  Huyên Pham,et al.  Optimisation et contrôle stochastique appliqués à la finance , 2007 .

[89]  H. Pham.  On the Smooth-Fit Property for One-Dimensional Optimal Switching Problem , 2004, math/0410285.

[90]  B. Øksendal,et al.  Applied Stochastic Control of Jump Diffusions , 2004, Universitext.