Linear Quadratic Regulation and Stabilization of Discrete-Time Systems With Delay and Multiplicative Noise

This paper is concerned with the long-standing problems of linear quadratic regulation (LQR) control and stabilization for a class of discrete-time stochastic systems involving multiplicative noises and input delay. These fundamental problems have attracted resurgent interests due to development of networked control systems. An explicit analytical expression is given for the optimal LQR controller. More specifically, the optimal LQR controller is shown to be a linear function of the conditional expectation of the state, with the feedback gain based on a Riccati-ZXL difference equation. It is also shown that the system is stabilizable in the mean-square sense if and only if an algebraic Riccati-ZXL equation has a particular solution. These results are based on a new technical tool, which establishes a non-homogeneous relationship between the state and the costate of this class of systems, and the introduction of a new Lyapunov function for the finite-horizon optimal control design.

[1]  E. Ostertag.  Linear Matrix Inequalities , 2011 .

[2]  U. G. Haussmann,et al.  Optimal Stationary Control with State Control Dependent Noise , 1971 .

[3]  M. Delfour.  The linear quadratic optimal control problem with delays in state and control variables: a state space approach , 1986 .

[4]  M. Athans,et al.  Further results on the uncertainty threshold principle , 1977 .

[5]  Masami Ito,et al.  A process-model control for linear systems with delay , 1981 .

[6]  Z. Artstein.  Linear systems with delayed controls: A reduction , 1982 .

[7]  Huanshui Zhang,et al.  Infinite Horizon LQ Optimal Control for Discrete-Time Stochastic Systems , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[8]  D. Naidu,et al.  Optimal Control Systems , 2018 .

[9]  E. Lee,et al.  The quadratic problem for systems with time delays , 1971 .

[10]  X. Chen,et al.  Discrete-time Indefinite LQ Control with State and Control Dependent Noises , 2002, J. Glob. Optim..

[11]  B. Larssen.  Dynamic programming in stochastic control of systems with delay , 2002 .

[12]  D. Kleinman,et al.  Optimal stationary control of linear systems with control-dependent noise , 1969 .

[13]  J. Bismut.  Linear Quadratic Optimal Stochastic Control with Random Coefficients , 1976 .

[14]  Xun Yu Zhou,et al.  Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls , 2000, IEEE Trans. Autom. Control..

[15]  Huanshui Zhang,et al.  Adapted and casual maximum principle and analytical solution to optimal control for stochastic multiplicative-noise systems with multiple input-delays , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[16]  Xun Yu Zhou,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II , 2000, SIAM J. Control. Optim..

[17]  Jean-Louis Thonnard,et al.  Improving the state estimation for optimal control of stochastic processes subject to multiplicative noise , 2011, Autom..

[18]  Huanshui Zhang,et al.  Linear quadratic regulation for linear time-varying systems with multiple input delays part I: discrete-time case , 2005, 2005 International Conference on Control and Automation.

[19]  João Pedro Hespanha,et al.  A Survey of Recent Results in Networked Control Systems , 2007, Proceedings of the IEEE.

[20]  Panos J. Antsaklis,et al.  Control and Communication Challenges in Networked Real-Time Systems , 2007, Proceedings of the IEEE.

[21]  E. Lee,et al.  Controller synthesis for linear systems with retarded state and control variables and quadratic cost , 1972 .

[22]  Tao Pang,et al.  A Stochastic Portfolio Optimization Model with Bounded Memory , 2011, Math. Oper. Res..

[23]  S. Joshi.  On Optimal Control of Linear Systems in the Presence of Multiplicative Noise , 1976, IEEE Transactions on Aerospace and Electronic Systems.

[24]  Zhen Wu,et al.  Maximum principle for the stochastic optimal control problem with delay and application , 2010, Autom..

[25]  Xi Chen,et al.  Solvability and asymptotic behavior of generalized Riccati equations arising in indefinite stochastic LQ controls , 2001, IEEE Trans. Autom. Control..

[26]  W. Wonham.  On a Matrix Riccati Equation of Stochastic Control , 1968 .

[27]  Bruno Sinopoli,et al.  Foundations of Control and Estimation Over Lossy Networks , 2007, Proceedings of the IEEE.

[28]  Minyue Fu,et al.  Optimal state estimation using randomly delayed measurements without time stamping , 2014 .

[29]  Huanshui Zhang,et al.  Stochastic control for Itô system with state transmission delay , 2015, 2015 American Control Conference (ACC).

[30]  Huanshui Zhang,et al.  Quantized stabilization for stochastic discrete‐time systems with multiplicative noises , 2013 .

[31]  L. Mirkin,et al.  H/sup /spl infin// control and estimation with preview-part II: fixed-size ARE solutions in discrete time , 2005, IEEE Transactions on Automatic Control.