Optimal Control of Diffusion Processes

In this chapter, we study optimization problems with diffusion processes for long-run average, finite-horizon, optimal stopping, and singular control. The value function for finite-horizon problems, or the potential function for long-run average, can be smooth or semi-smooth (both one-sided first-order derivatives exist, but not equal). Explicit optimality conditions are derived at both smooth and semi-smooth points. This extends the famous Hamilton-Jacobi-Bellman (HJB) equations from smooth value functions to semi-smooth value functions, which cover the degenerate diffusion processes. Viscosity solution is not used. The performance-difference formula is based on the Ito-Tanaka formula for semi-smooth functions, which involves local time in \([t, t+ dt]\) with a mean of the order of \(\sqrt{dt}\). We also show that under some conditions, the semi-smoothness of the value (or potential) functions can simply be ignored.

[1]  A. Skorokhod.  Stochastic Equations for Diffusion Processes in a Bounded Region , 1961 .

[2]  Note on continuous additive functional of the 1-dimensional Brownian path , 1963 .

[3]  P. Meyer,et al.  Introduction et notations generales , 1976 .

[4]  Albert T. J. Wang.  Generalized Ito's formula and additive functionals of Brownian motion , 1977 .

[5]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  H. Witsenhausen,et al.  Some solvable stochastic control problemst , 1980 .

[7]  L. Rogers.  Stochastic differential equations and diffusion processes: Nobuyuki Ikeda and Shinzo Watanabe North-Holland, Amsterdam, 1981, xiv + 464 pages, Dfl.175.00 , 1982 .

[8]  R. Bass.  Joint continuity and representations of additive functionals of d-dimensional Brownian motion , 1984 .

[9]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[10]  B. Øksendal.  Stochastic differential equations : an introduction with applications , 1987 .

[11]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[12]  S. Jacka.  Optimal Stopping and the American Put , 1991 .

[13]  Richard F. Bass,et al.  Local times on curves and uniform invariance principles , 1992 .

[14]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[15]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[16]  Kristin Reikvam.  Viscosity solutions of optimal stopping problems , 1997 .

[17]  Distribution of Brownian Local Time on Curves , 1998 .

[18]  F. Klebaner.  Introduction To Stochastic Calculus With Applications , 1999 .

[19]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[20]  Agnès Sulem,et al.  Dynamic Optimization of Long-Term Growth Rate for a Portfolio with Transaction Costs and Logarithmic Utility , 2001 .

[21]  Goran Peskir,et al.  A Change-of-Variable Formula with Local Time on Curves , 2005 .

[22]  V. Linetsky.  On the transition densities for reflected diffusions , 2005, Advances in Applied Probability.

[23]  G. Peskir.  A Change-of-Variable Formula with Local Time on Curves , 2005 .

[24]  Xi-Ren Cao,et al.  Stochastic learning and optimization - A sensitivity-based approach , 2007, Annu. Rev. Control..

[25]  S. Shreve,et al.  An explicit formula for the Skorokhod map on [0,a]. , 2007, 0710.2977.

[26]  Søren Asmussen,et al.  Loss Rates for Lévy Processes with Two Reflecting Barriers , 2007, Math. Oper. Res..

[27]  Hanqing Jin,et al.  BEHAVIORAL PORTFOLIO SELECTION IN CONTINUOUS TIME , 2007, 0709.2830.

[28]  Takashi Tamura.  Maximization of the long-term growth rate for a portfolio with fixed and proportional transaction costs , 2008, Advances in Applied Probability.

[29]  A. Bensoussan,et al.  Singular control and impulse control: A common approach , 2009 .

[30]  S. Shreve,et al.  Methods of Mathematical Finance , 2010 .

[31]  S. Asmussen,et al.  Local Time Asymptotics for Centered Lévy Processes with Two-Sided Reflection , 2011 .

[32]  Harold J. Kushner,et al.  Probability Methods for Approximations in Stochastic Control and for Elliptic Equations , 2012 .

[33]  Peter W. Glynn,et al.  On the Dynamics of Semimartingales with Two Reflecting Barriers , 2013, J. Appl. Probab..

[34]  P. Glynn,et al.  On the Dynamics of Semimartingales with Two Reflecting Barriers , 2013, Journal of Applied Probability.

[35]  Rob J. Wang,et al.  Central Limit Theorems and Large Deviations for Additive Functionals of Reflecting Diffusion Processes , 2013, 1307.1574.

[36]  Peter W. Glynn,et al.  Levy Processes with Two-Sided Reflection , 2015 .

[37]  M. Forde,et al.  Large deviations for the boundary local time of doubly reflected Brownian motion , 2014, 1409.1972.

[38]  John C Maxwell,et al.  PDF) Mscit Question Bank With Answer (PDF) The 17 Essential Qualities Of A Team Player Becoming Kind Person Every Wants , 2016 .

[39]  Xi-Ren Cao,et al.  Optimality Conditions for Long-Run Average Rewards With Underselectivity and Nonsmooth Features , 2017, IEEE Transactions on Automatic Control.

[40]  Xi-Ren Cao,et al.  Relative Time and Stochastic Control With Non-Smooth Features , 2017, IEEE Transactions on Automatic Control.

[41]  Xi-Ren Cao,et al.  SENSITIVITY ANALYSIS OF NONLINEAR BEHAVIOR WITH DISTORTED PROBABILITY , 2013 .

[42]  Xi-Ren Cao,et al.  Stochastic Feedback Control With One-Dimensional Degenerate Diffusions and Nonsmooth Value Functions , 2017, IEEE Transactions on Automatic Control.

[43]  B. Øksendal,et al.  Applied Stochastic Control of Jump Diffusions , 2004, Universitext.