Discrete time mean-field stochastic linear-quadratic optimal control problems

This paper firstly presents necessary and sufficient conditions for the solvability of discrete time, mean-field, stochastic linear-quadratic optimal control problems. Secondly, the optimal control within a class of linear feedback controls is investigated using a matrix dynamical optimization method. Thirdly, by introducing several sequences of bounded linear operators, the problem is formulated as an operator stochastic linear-quadratic optimal control problem. By the kernel-range decomposition representation of the expectation operator and its pseudo-inverse, the optimal control is derived using solutions to two algebraic Riccati difference equations. Finally, by completing the square, the two Riccati equations and the optimal control are also obtained.

[1]  Yulin Huang,et al.  Stochastic H2/Hinfinity control for discrete-time systems with state and disturbance dependent noise , 2007, Autom..

[2]  Ji-Feng Zhang,et al.  Mean Field Games for Large-Population Multiagent Systems with Markov Jump Parameters , 2012, SIAM J. Control. Optim..

[3]  Juan Li,et al.  Stochastic maximum principle in the mean-field controls , 2012, Autom..

[4]  Jiongmin Yong,et al.  Linear-Quadratic Optimal Control Problems for Mean-Field Stochastic Differential Equations , 2013, SIAM J. Control. Optim..

[5]  Leiba Rodman,et al.  Algebraic Riccati equations , 1995 .

[6]  J. Lions.  Optimal Control of Systems Governed by Partial Differential Equations , 1971 .

[7]  Tao Li,et al.  Decentralized tracking-type games for multi-agent systems with coupled ARX models: Asymptotic Nash equilibria , 2008, Autom..

[8]  Boualem Djehiche,et al.  A General Stochastic Maximum Principle for SDEs of Mean-field Type , 2011 .

[9]  D. Dawson.  Critical dynamics and fluctuations for a mean-field model of cooperative behavior , 1983 .

[10]  T. Chan,et al.  Dynamics of the McKean-Vlasov equation , 1994 .

[11]  Boualem Djehiche,et al.  Mean-Field Backward Stochastic Differential Equations . A Limit Approach ∗ , 2007 .

[12]  Tobias Damm.  On detectability of stochastic systems , 2007, Autom..

[13]  Minyi Huang,et al.  Linear-Quadratic-Gaussian Mixed Games with Continuum-Parametrized Minor Players , 2012, SIAM J. Control. Optim..

[14]  J. Gärtner.  On the McKean‐Vlasov Limit for Interacting Diffusions , 1988 .

[15]  M. Kac.  Foundations of Kinetic Theory , 1956 .

[16]  P. Caines,et al.  Social optima in mean field LQG control: Centralized and decentralized strategies , 2009 .

[17]  J. Yong.  Linear-Quadratic Optimal Control Problems for Mean-Field Stochastic Differential Equations --- Time-Consistent Solutions , 2013, 1304.3964.

[18]  X. Chen,et al.  Discrete-time Indefinite LQ Control with State and Control Dependent Noises , 2002, J. Glob. Optim..

[19]  Michael Athans,et al.  The Matrix Minimum Principle , 1967, Inf. Control..

[20]  H. Abou-Kandil,et al.  Matrix Riccati Equations in Control and Systems Theory , 2003, IEEE Transactions on Automatic Control.

[21]  Bor-Sen Chen,et al.  On stabilizability and exact observability of stochastic systems with their applications , 2023, Autom..

[22]  Jiongmin Yong,et al.  Two-person zero-sum linear quadratic stochastic differential games by a Hilbert space method , 2006 .

[23]  S. Peng,et al.  Mean-field backward stochastic differential equations and related partial differential equations , 2007, 0711.2167.

[24]  J. Yong.  A Linear-Quadratic Optimal Control Problem for Mean-Field Stochastic Differential Equations , 2011, 1110.1564.

[25]  N. U. Ahmed,et al.  Nonlinear Diffusion Governed by McKean--Vlasov Equation on Hilbert Space and Optimal Control , 2007, SIAM J. Control. Optim..

[26]  Y. Huang,et al.  Stochastic H2/H8 control for discrete-time systems with state and disturbance dependent noise. , 2007 .

[27]  Carlos S. Kubrusly.  Mean Square Stability for Discrete Bounded Linear Systems in Hilbert Space , 1985 .

[28]  Daniel Andersson,et al.  A Maximum Principle for SDEs of Mean-Field Type , 2011 .

[29]  Alain Bensoussan,et al.  Representation and Control of Infinite Dimensional Systems, 2nd Edition , 2007, Systems and control.

[30]  T. Morozan.  Stability of stochastic discrete systems , 1968 .

[31]  L. Berkovitz.  Optimal Control Theory , 1974 .

[32]  D. Whittaker,et al.  A Course in Functional Analysis , 1991, The Mathematical Gazette.

[33]  Joe Brewer,et al.  Kronecker products and matrix calculus in system theory , 1978 .

[34]  Minyi Huang,et al.  Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria , 2007, IEEE Transactions on Automatic Control.

[35]  Tao Li,et al.  Asymptotically Optimal Decentralized Control for Large Population Stochastic Multiagent Systems , 2008, IEEE Transactions on Automatic Control.

[36]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[37]  A. Sznitman.  Topics in propagation of chaos , 1991 .

[38]  D. Crisan,et al.  Approximate McKean–Vlasov representations for a class of SPDEs , 2005, math/0510668.

[39]  S. R. Caradus,et al.  Operator theory of the pseudo-inverse , 1974 .

[40]  P. Lions,et al.  Mean field games , 2007 .

[41]  H. Wimmer.  The set of positive semidefinite solutions of the algebraic Riccati equation of discrete-time optimal control , 1996, IEEE Trans. Autom. Control..

[42]  Bernt Øksendal,et al.  A mean-field stochastic maximum principle via Malliavin calculus , 2012 .

[43]  M. Hp.  A class of markov processes associated with nonlinear parabolic equations. , 1966 .

[44]  Weihai Zhang,et al.  On the observability and detectability of linear stochastic systems with Markov jumps and multiplicative noise , 2010, J. Syst. Sci. Complex..

[45]  Carl Graham,et al.  McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets , 1992 .

[46]  H. McKean,et al.  A CLASS OF MARKOV PROCESSES ASSOCIATED WITH NONLINEAR PARABOLIC EQUATIONS , 1966, Proceedings of the National Academy of Sciences of the United States of America.

[47]  Weihai Zhang,et al.  Infinite Horizon H2/H∞ Control for Discrete-Time Time-Varying Markov Jump Systems with Multiplicative Noise* , 2011 .

[48]  Vlad Ionescu,et al.  Generalized Riccati theory and robust control , 1999 .

[49]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.