A Simple Proof of Indefinite Linear-Quadratic Stochastic Optimal Control With Random Coefficients

We consider the indefinite linear-quadratic optimal control problem for stochastic systems with random coefficients, where the corresponding cost parameters need not be definite matrices. Although the solution to this problem was obtained previously via the stochastic maximum principle or by solving the stochastic Hamilton–Jacobi–Bellman (HJB) equation, our approach is simple and direct. Specifically, we develop a direct approach, also known as the completion of squares method, to characterize the explicit optimal solution and the optimal cost. The corresponding optimal solution is linear characterized in terms of the stochastic Riccati differential equation (SRDE) and the linear backward stochastic differential equation (BSDE). In our approach, the completion of squares method, together with the SRDE and the BSDE, allows us to construct an equivalent cost functional that is quadratic in the control $u$, provided that an additional positive definiteness condition in terms of the SRDE holds. Then, the optimal control and the associated cost can be obtained by eliminating the quadratic term of $u$ in the equivalent cost functional. The additional positive definiteness condition is induced due to the indefiniteness of the cost parameters and the dependence of the control on the diffusion term. We verify the optimal solution by solving the stochastic HJB equation.

[1]  J. Yong,et al.  Stochastic Linear Quadratic Optimal Control Problems , 2001 .

[2]  Haiyang Wang,et al.  Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation , 2015 .

[3]  Rathinasamy Sakthivel,et al.  Stabilization of stochastic differential equations driven by G-Brownian motion with feedback control based on discrete-time state observation , 2018, Autom..

[4]  Chenshuping,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs , 1998 .

[5]  Xun Yu Zhou,et al.  Existence of Solutions to a Class of Indefinite Stochastic Riccati Equations , 2013, SIAM J. Control. Optim..

[6]  Hiroaki Morimoto Stochastic Control and Mathematical Modeling: Viscosity Solutions of Hamilton–Jacobi–Bellman Equations , 2010 .

[7]  Michael Kohlmann,et al.  Multidimensional Backward Stochastic Riccati Equations and Applications , 2002, SIAM J. Control. Optim..

[8]  Xun Yu Zhou,et al.  Indefinite Stochastic Riccati Equations , 2003, SIAM J. Control. Optim..

[9]  Patrizio Colaneri,et al.  Markov Jump Linear Systems with switching transition rates: Mean square stability with dwell-time , 2010, Autom..

[10]  J. Bismut Linear Quadratic Optimal Stochastic Control with Random Coefficients , 1976 .

[11]  Andrew E. B. Lim,et al.  Decentralized Control of a Stochastic Multi-Agent Queueing System , 2012, IEEE Transactions on Automatic Control.

[12]  Peter E. Caines,et al.  Uplink power adjustment in wireless communication systems: a stochastic control analysis , 2004, IEEE Transactions on Automatic Control.

[13]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[14]  Jiongmin Yong,et al.  Stochastic Linear-Quadratic Optimal Control Problems with Random Coefficients: Closed-Loop Representation of Open-Loop Optimal Controls , 2018 .

[15]  Xun Yu Zhou,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II , 2000, SIAM J. Control. Optim..

[16]  X. Zhou,et al.  Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework , 2000 .

[17]  X. Zhou,et al.  Indefinite stochastic linear quadratic control and generalized differential Riccati equation , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[18]  Shanjian Tang,et al.  Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients , 2014, SIAM J. Control. Optim..

[19]  Jun Moon,et al.  A Sufficient Condition for Linear-Quadratic Stochastic Zero-Sum Differential Games for Markov Jump Systems , 2019, IEEE Transactions on Automatic Control.

[20]  M. James Controlled markov processes and viscosity solutions , 1994 .

[21]  Shige Peng,et al.  Stochastic Hamilton-Jacobi-Bellman equations , 1992 .

[22]  Andrew E. B. Lim,et al.  Linear-Quadratic Control of Backward Stochastic Differential Equations , 2001, SIAM J. Control. Optim..

[23]  Yang Shi,et al.  Analysis and synthesis for a class of stochastic switching systems against delayed mode switching: A framework of integrating mode weights , 2019, Autom..

[24]  Tamer Başar,et al.  Risk-sensitive control of Markov jump linear systems: Caveats and difficulties , 2017 .

[25]  Tyrone E. Duncan,et al.  Linear-Quadratic Fractional Gaussian Control , 2013, SIAM J. Control. Optim..

[26]  Zhiyong Yu,et al.  Solvability of indefinite stochastic Riccati equations and linear quadratic optimal control problems , 2014, Syst. Control. Lett..

[27]  Zhiwei Gao,et al.  Observer-based controller design for stochastic descriptor systems with Brownian motions , 2013, Autom..

[28]  Michael Kohlmann,et al.  Minimization of Risk and Linear Quadratic Optimal Control Theory , 2003, SIAM J. Control. Optim..

[29]  Hamidou Tembine,et al.  A stochastic maximum principle for risk-sensitive mean-field-type control , 2014, 53rd IEEE Conference on Decision and Control.

[30]  Jianfeng Zhang,et al.  Backward Stochastic Differential Equations , 2017 .

[31]  N. Touzi Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE , 2012 .

[32]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[33]  Arman C. Kizilkale,et al.  Collective Target Tracking Mean Field Control for Markovian Jump-Driven Models of Electric Water Heating Loads , 2014 .

[34]  Tyrone E. Duncan,et al.  Linear-Exponential-Quadratic Gaussian Control , 2013, IEEE Transactions on Automatic Control.

[35]  Xiongzhi Chen Brownian Motion and Stochastic Calculus , 2008 .

[36]  Ian R. Petersen,et al.  Absolute Stabilization and Minimax Optimal Control of Uncertain Systems with Stochastic Uncertainty , 1997 .

[37]  Shanjian Tang,et al.  General Linear Quadratic Optimal Stochastic Control Problems with Random Coefficients: Linear Stochastic Hamilton Systems and Backward Stochastic Riccati Equations , 2003, SIAM J. Control. Optim..

[38]  Kai Du,et al.  Solvability Conditions for Indefinite Linear Quadratic Optimal Stochastic Control Problems and Associated Stochastic Riccati Equations , 2014, SIAM J. Control. Optim..