Solution to stochastic LQ control problem for Itô systems with state delay or input delay

Abstract This paper is concerned with the linear quadratic (LQ) control problem for Ito stochastic systems with state delay or input delay. The main contribution is to give the explicit optimal LQ controllers for Ito stochastic systems with input delay and with state delay respectively. For the case of state delay, the optimal LQ controller is a linear function of the state at the current time and the past time. For the case of input delay, the optimal LQ controller is a linear function of the state and the past inputs. The key technology is to define the value function and complete the square based on the coupled differential equations.

[1]  Huanshui Zhang,et al.  LQ control for Itô-type stochastic systems with input delays , 2013, Autom..

[2]  M. Delfour.  The linear quadratic optimal control problem with delays in state and control variables: a state space approach , 1986 .

[3]  X. Mao,et al.  Stochastic Differential Equations and Applications , 1998 .

[4]  Huanshui Zhang,et al.  Optimal Control and Stabilization for Networked Control Systems With Packet Dropout and Input Delay , 2017, IEEE Transactions on Circuits and Systems II: Express Briefs.

[5]  Huanshui Zhang,et al.  Consensus problems for discrete-time agents with communication delay , 2017 .

[6]  Bernt Øksendal,et al.  Some Solvable Stochastic Control Problems With Delay , 2000 .

[7]  Zhen Wu,et al.  Delayed Stochastic Linear-Quadratic Control Problem and Related Applications , 2012, J. Appl. Math..

[8]  John B. Moore,et al.  Indefinite Stochastic Linear Quadratic Control and Generalized Differential Riccati Equation , 2002, SIAM J. Control. Optim..

[9]  B. Larssen.  Dynamic programming in stochastic control of systems with delay , 2002 .

[10]  Bruno Sinopoli,et al.  Foundations of Control and Estimation Over Lossy Networks , 2007, Proceedings of the IEEE.

[11]  Zhenhua Wang,et al.  Consensusability of multi-agent systems with time-varying communication delay , 2014, Syst. Control. Lett..

[12]  Panos J. Antsaklis,et al.  Control and Communication Challenges in Networked Real-Time Systems , 2007, Proceedings of the IEEE.

[13]  Minyue Fu,et al.  Linear Quadratic Regulation and Stabilization of Discrete-Time Systems With Delay and Multiplicative Noise , 2015, IEEE Transactions on Automatic Control.

[14]  R. C. Merton,et al.  Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case , 1969 .

[15]  Xun Li,et al.  Forward-backward linear quadratic stochastic optimal control problem with delay , 2012, Syst. Control. Lett..

[16]  Huanshui Zhang,et al.  Discrete-Time LQG Control With Input Delay and Multiplicative Noise , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[17]  H. Banks,et al.  Optimal control of linear time-delay systems , 1969 .

[18]  Huanshui Zhang,et al.  Control for Itô Stochastic Systems With Input Delay , 2017, IEEE Transactions on Automatic Control.

[19]  S. Peng,et al.  Adapted solution of a backward stochastic differential equation , 1990 .

[20]  O. J. M. Smith,et al.  A controller to overcome dead time , 1959 .

[21]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .