Approximate Infinite-Horizon Optimal Control for Stochastic Systems

The policy of an optimal control problem for nonlinear stochastic systems can be characterized by a second-order partial differential equation for which solutions are not readily available. In this paper we provide a systematic method for obtaining approximate solutions for the infinite-horizon optimal control problem in the stochastic framework. The method is demonstrated on an illustrative numerical example in which the control effort is not weighted, showing that the technique is able to deal with one of the most striking features of stochastic optimal control.

[1]  R. C. Merton,et al.  Optimum Consumption and Portfolio Rules in a Continuous-Time Model* , 1975 .

[2]  Jennifer F. Reinganum.  On the diffusion of new technology: A game theoretic approach , 1981 .

[3]  Mario Sassano,et al.  Autonomous collision avoidance for wheeled mobile robots using a differential game approach , 2017, Eur. J. Control.

[4]  R. C. Merton,et al.  Optimum consumption and portfolio rules in a continuous - time model Journal of Economic Theory 3 , 1971 .

[5]  Xun Yu Zhou,et al.  Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls , 2000, IEEE Trans. Autom. Control..

[6]  F. Hohn,et al.  Production Planning Over Time and the Nature of the Expectation and Planning Horizon , 1955 .

[7]  R. E. Kalman,et al.  Contributions to the Theory of Optimal Control , 1960 .

[8]  Alessandro Astolfi,et al.  Approximate finite-horizon optimal control for input-affine nonlinear systems with input constraints , 2014 .

[9]  R. C. Merton,et al.  Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case , 1969 .

[10]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  M. Sassano,et al.  Approximate solutions to a class of nonlinear differential games using a shared dynamic extension , 2013, ECC.

[12]  S. Zacks,et al.  Introduction to stochastic differential equations , 1988 .

[13]  Alessandro Astolfi,et al.  Approximate finite-horizon optimal control without PDE's , 2011, IEEE Conference on Decision and Control and European Control Conference.

[14]  Alessandro Astolfi,et al.  Approximate finite-horizon optimal control without PDEs , 2011, Syst. Control. Lett..

[15]  Alessandro Astolfi,et al.  Approximate solutions to a class of nonlinear differential games using a shared dynamic extension , 2012, 2013 European Control Conference (ECC).

[16]  Harold J. Kushner,et al.  Optimal stochastic control , 1962 .

[17]  M. Weitzman,et al.  FUNDING CRITERIA FOR RESEARCH, DEVELOPMENT, AND EXPLORATION PROJECTS , 1981 .

[18]  Alessandro Astolfi,et al.  Dynamic Approximate Solutions of the HJ Inequality and of the HJB Equation for Input-Affine Nonlinear Systems , 2012, IEEE Transactions on Automatic Control.

[19]  Xun Yu Zhou,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II , 2000, SIAM J. Control. Optim..

[20]  Mario Lefebvre.  Optimal stochastic control of a reservoir , 1995 .

[21]  Alessandro Astolfi,et al.  A Differential Game Approach to Multi-agent Collision Avoidance , 2017, IEEE Transactions on Automatic Control.

[22]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[23]  R. Newcomb.  VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS , 2010 .

[24]  Alessandro Astolfi,et al.  A constructive differential game approach to collision avoidance in multi-agent systems , 2014, 2014 American Control Conference.

[25]  Alessandro Astolfi,et al.  Constructive $\epsilon$-Nash Equilibria for Nonzero-Sum Differential Games , 2015, IEEE Transactions on Automatic Control.