LaSalle-Type Theorem and Its Applications to Infinite Horizon Optimal Control of Discrete-Time Nonlinear Stochastic Systems

Based on discrete martingale theory, the LaSalle-type theorem for general discrete-time stochastic systems is obtained and the almost sure stability is in turn discussed. As applications, infinite horizon nonlinear optimal regulator is investigated, and a dynamical programming equation called the Hamilton-Jacobi-Bellman equation is also derived for discrete-time nonlinear stochastic optimal control.

[1]  Kok Lay Teo,et al.  An integrated optimal control algorithm for discrete-time nonlinear stochastic system , 2010, Int. J. Control.

[2]  Feiqi Deng,et al.  Robust exponential stability and delayed-state-feedback stabilization of uncertain impulsive stochastic systems with time-varying delay , 2012 .

[3]  Richard Bellman,et al.  On moment behavior of a class of stochastic difference equations , 1972 .

[4]  Michael V. Basin,et al.  Discrete-time optimal control for stochastic nonlinear polynomial systems , 2014, Int. J. Gen. Syst..

[5]  Weihai Zhang,et al.  State Feedback HINFINITY Control for a Class of Nonlinear Stochastic Systems , 2006, SIAM J. Control. Optim..

[6]  Huanshui Zhang,et al.  Infinite horizon linear quadratic optimal control for discrete‐time stochastic systems , 2008 .

[7]  Xuerong Mao,et al.  Stochastic Versions of the LaSalle Theorem , 1999 .

[8]  Cónall Kelly,et al.  Almost sure instability of the equilibrium solution of a Milstein-type stochastic difference equation , 2013, Comput. Math. Appl..

[9]  Xuerong Mao,et al.  The improved LaSalle-type theorems for stochastic functional differential equations , 2006 .

[10]  Xiangyun Lin,et al.  A Maximum Principle for Optimal Control of Discrete-Time Stochastic Systems With Multiplicative Noise , 2015, IEEE Transactions on Automatic Control.

[11]  Fuke Wu,et al.  The LaSalle-type theorem for neutral stochastic functional differential equations with infinite delay , 2011 .

[12]  M. Fragoso,et al.  Stability Results for Discrete-Time Linear Systems with Markovian Jumping Parameters , 1993 .

[13]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[14]  O. Kallenberg.  Foundations of Modern Probability , 2021, Probability Theory and Stochastic Modelling.

[15]  Eduardo Liz,et al.  Stability of non-autonomous difference equations: simple ideas leading to useful results , 2011 .

[16]  J. Geromel,et al.  A new discrete-time robust stability condition , 1999 .

[17]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Weihai Zhang,et al.  Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems , 2015 .

[19]  Takeshi Taniguchi.  Stability theorems of stochastic difference equations , 1990 .

[20]  Edgar N. Sánchez,et al.  Inverse optimal control for discrete-time stochastic nonlinear systems stabilization , 2013, 2013 American Control Conference.

[21]  Feiqi Deng,et al.  LaSalle-Type Theorems for General Nonlinear Stochastic Functional Differential Equations by Multiple Lyapunov Functions , 2014 .

[22]  J. Hurt.  Some Stability Theorems for Ordinary Difference Equations , 1967 .

[23]  J. P. Lasalle.  The stability and control of discrete processes , 1986 .

[24]  F. Deng,et al.  Global exponential stability of impulsive stochastic functional differential systems , 2010 .

[25]  J. P. Lasalle.  Stability theory for ordinary differential equations. , 1968 .