Stochastic Linear-Quadratic Control via Semidefinite Programming

We study stochastic linear-quadratic (LQ) optimal control problems over an infinite time horizon, allowing the cost matrices to be indefinite. We develop a systematic approach based on semidefinite programming (SDP). A central issue is the stability of the feedback control; and we show this can be effectively examined through the complementary duality of the SDP. Furthermore, we establish several implication relations among the SDP complementary duality, the (generalized) Riccati equation, and the optimality of the LQ control problem. Based on these relations, we propose a numerical procedure that provides a thorough treatment of the LQ control problem via primal-dual SDP: it identifies a stabilizing feedback control that is optimal or determines that the problem possesses no optimal solution. For the latter case, we develop an $\epsilon$-approximation scheme that is asymptotically optimal.

[1]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[2]  B. Anderson,et al.  Optimal control: linear quadratic methods , 1990 .

[3]  Z. Luo,et al.  Conic convex programming and self-dual embedding , 1998 .

[4]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[5]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[6]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[7]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[8]  Mark H. A. Davis.  Linear estimation and stochastic control , 1977 .

[9]  Farid Alizadeh,et al.  Interior Point Methods in Semidefinite Programming with Applications to Combinatorial Optimization , 1995, SIAM J. Optim..

[10]  R. E. Kalman,et al.  Contributions to the Theory of Optimal Control , 1960 .

[11]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[12]  W. Wonham.  On the Separation Theorem of Stochastic Control , 1968 .

[13]  Shuzhong Zhang,et al.  Duality Results for Conic Convex Programming , 1997 .

[14]  A. Albert.  Conditions for Positive and Nonnegative Definiteness in Terms of Pseudoinverses , 1969 .

[15]  A. Bensoussan.  Lectures on stochastic control , 1982 .

[16]  Xun Yu Zhou,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II , 2000, SIAM J. Control. Optim..

[17]  Xun Yu Zhou,et al.  Relationship Between Backward Stochastic Differential Equations and Stochastic Controls: A Linear-Quadratic Approach , 2000, SIAM J. Control. Optim..

[18]  X. Zhou,et al.  Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework , 2000 .