Maximum principle for discrete-time stochastic optimal control problem and stochastic game

This paper is first concerned with one kind of discrete-time stochastic optimal control problem with convex control domains, for which necessary condition in the form of Pontryagin's maximum principle and sufficient condition of optimality are derived. The results are then extended to two kinds of discrete-time stochastic games. Two illustrative examples are studied, for which the explicit optimal strategies are given. This paper establishes a rigorous version of discrete-time stochastic maximum principle in a clear and concise way and paves a road for further related topics.

[1]  Weihai Zhang,et al.  Infinite horizon linear quadratic differential games for discrete-time stochastic systems , 2012 .

[2]  Domenico D'Alessandro,et al.  Discrete-Time Optimal Control with Control-Dependent Noise and Generalized Riccati Difference Equations , 1998, Autom..

[3]  Oswaldo Luiz V. Costa,et al.  Optimal mean-variance control for discrete-time linear systems with Markovian jumps and multiplicative noises , 2012, Autom..

[4]  Xu Zhang,et al.  Second-Order Necessary Conditions for Stochastic Optimal Control Problems , 2018, SIAM Rev..

[5]  Xun Li,et al.  Discrete-time mean-field Stochastic linear-quadratic optimal control problems, II: Infinite horizon case , 2015, Autom..

[6]  Samuel N. Cohen,et al.  A general theory of finite state Backward Stochastic Difference Equations , 2008, 0810.4957.

[7]  Khanh Pham,et al.  Discrete-Time Linear-Quadratic Dynamic Games , 2010 .

[8]  Bernt Øksendal,et al.  Maximum Principle for Stochastic Differential Games with Partial Information , 2008 .

[9]  Xiangyun Lin,et al.  A Maximum Principle for Optimal Control of Discrete-Time Stochastic Systems With Multiplicative Noise , 2015, IEEE Transactions on Automatic Control.

[10]  Qi Lu,et al.  General Pontryagin-Type Stochastic Maximum Principle and Backward Stochastic Evolution Equations in Infinite Dimensions , 2012, 1204.3275.

[11]  S. Peng.  A general stochastic maximum principle for optimal control problems , 1990 .

[12]  Yulin Huang,et al.  Stochastic H2/Hinfinity control for discrete-time systems with state and disturbance dependent noise , 2007, Autom..

[13]  Robert J. Elliott,et al.  A Stochastic Maximum Principle for a Markov Regime-Switching Jump-Diffusion Model and Its Application to Finance , 2012, SIAM J. Control. Optim..

[14]  H. Halkin.  A Maximum Principle of the Pontryagin Type for Systems Described by Nonlinear Difference Equations , 1966 .

[15]  Zhiyong Yu,et al.  A Pontryagin's Maximum Principle for Non-Zero Sum Differential Games of BSDEs with Applications , 2010, IEEE Transactions on Automatic Control.

[16]  Zhen Wu,et al.  Maximum Principle for Backward Doubly Stochastic Control Systems with Applications , 2010, SIAM J. Control. Optim..

[17]  Weihai Zhang,et al.  Stability and stabilization of nonlinear discrete‐time stochastic systems , 2019, International Journal of Robust and Nonlinear Control.

[18]  Robert J. Elliott,et al.  Backward Stochastic Difference Equations and Nearly Time-Consistent Nonlinear Expectations , 2011, SIAM J. Control. Optim..

[19]  Debasish Chatterjee,et al.  Discrete Time Pontryagin Maximum Principle Under State-Action-Frequency Constraints , 2019, IEEE Transactions on Automatic Control.

[20]  Rufus Isaacs,et al.  Differential Games , 1965 .

[21]  Shaolin Ji,et al.  Maximum principle for stochastic optimal control problem of forward–backward stochastic difference systems , 2018, 1812.11283.

[22]  Qingxin Meng,et al.  A Maximum Principle for Optimal Control of Stochastic Evolution Equations , 2013, SIAM J. Control. Optim..

[23]  Duan Li,et al.  Cost smoothing in discrete-time linear-quadratic control , 1997, Autom..

[24]  Huanshui Zhang,et al.  Optimal control for stochastic discrete-time systems with multiple input-delays , 2012, Proceedings of the 10th World Congress on Intelligent Control and Automation.

[25]  Xun Li,et al.  Discrete time mean-field stochastic linear-quadratic optimal control problems , 2013, Autom..

[26]  Li Chen,et al.  Maximum Principle for Nonzero-Sum Stochastic Differential Game With Delays , 2015, IEEE Transactions on Automatic Control.

[27]  Zhen Wu,et al.  A general maximum principle for optimal control of forward-backward stochastic systems , 2013, Autom..

[28]  Andrew E. B. Lim,et al.  Discrete time LQG controls with control dependent noise , 1999 .

[29]  Eddie C.M. Hui,et al.  Maximum principle for differential games of forward–backward stochastic systems with applications , 2012 .

[30]  X. Chen,et al.  Discrete-time Indefinite LQ Control with State and Control Dependent Noises , 2002, J. Glob. Optim..