Stochastic Linear-Quadratic Control with Conic Control Constraints on an Infinite Time Horizon

This paper is concerned with a stochastic linear-quadratic (LQ) control problem in the infinite time horizon where the control is constrained in a given, arbitrary closed cone, the cost weighting matrices are allowed to be indefinite, and the state is scalar-valued. First, the (mean-square, conic) stabilizability of the system is defined, which is then characterized by a set of simple conditions involving linear matrix inequalities (LMIs). Next, the issue of well-posedness of the underlying optimal LQ control, which is necessitated by the indefiniteness of the problem, is addressed in great detail, and necessary and sufficient conditions of the well-posedness are presented. On the other hand, to address the LQ optimality two new algebraic equations a la Riccati, called extended algebraic Riccati equations (EAREs), along with the notion of their stabilizing solutions, are introduced for the first time. Optimal feedback control as well as the optimal value are explicitly derived in terms of the stabilizing solutions to the EAREs. Moreover, several cases when the stabilizing solutions do exist are discussed and algorithms of computing the solutions are presented. Finally, numerical examples are provided to illustrate the theoretical results established.

[1]  J. Yong,et al.  Stochastic Linear Quadratic Optimal Control Problems , 2001 .

[2]  Shuzhong Zhang,et al.  On Cones of Nonnegative Quadratic Functions , 2003, Math. Oper. Res..

[3]  J. Bismut.  Linear Quadratic Optimal Stochastic Control with Random Coefficients , 1976 .

[4]  James B. Rawlings,et al.  Constrained linear quadratic regulation , 1998, IEEE Trans. Autom. Control..

[5]  D. Jacobson,et al.  Stabilization with conic control constraint set , 1979 .

[6]  Xun Yu Zhou,et al.  Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls , 2000, IEEE Trans. Autom. Control..

[7]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[8]  R. Brammer.  Controllability in Linear Autonomous Systems with Positive Controllers , 1972 .

[9]  Andrew E. B. Lim,et al.  Dynamic Mean-Variance Portfolio Selection with No-Shorting Constraints , 2001, SIAM J. Control. Optim..

[10]  Stephen L. Campbell,et al.  On positive controllers and linear quadratic optimal control problems , 1982 .

[11]  S. Saperstone.  Global Controllability of Linear Systems with Positive Controls , 1973 .

[12]  David D. Yao,et al.  Stochastic Linear-Quadratic Control via Semidefinite Programming , 2001, SIAM J. Control. Optim..

[13]  G. Kallianpur.  Stochastic differential equations and diffusion processes , 1981 .

[14]  Xun Yu Zhou,et al.  Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II , 2000, SIAM J. Control. Optim..

[15]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[16]  X. Zhou,et al.  Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework , 2000 .

[17]  W. P. M. H. Heemels,et al.  Positive feedback stabilization of centrifugal compressor surge , 2002, Autom..

[18]  W. Wonham.  Optimal Stationary Control of a Linear System with State-Dependent Noise , 1967 .

[19]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[20]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[21]  Andrew E. B. Lim,et al.  Mean-Variance Portfolio Selection with Random Parameters in a Complete Market , 2002, Math. Oper. Res..

[22]  R. E. Kalman,et al.  Contributions to the Theory of Optimal Control , 1960 .

[23]  Michael Heymann,et al.  Controllability of linear systems with positive controls: Geometric considerations , 1975 .

[24]  W. D. Collins,et al.  The Numerical Solution of the Linear Regulator Problem with Positive Control , 1988 .

[25]  J. Yorke,et al.  Controllability of linear oscillatory systems using positive controls , 1971 .

[26]  D. Jacobson,et al.  Control with conic control constraint set , 1978 .

[27]  L. Rogers.  Stochastic differential equations and diffusion processes: Nobuyuki Ikeda and Shinzo Watanabe North-Holland, Amsterdam, 1981, xiv + 464 pages, Dfl.175.00 , 1982 .

[28]  Xi Chen,et al.  Solvability and asymptotic behavior of generalized Riccati equations arising in indefinite stochastic LQ controls , 2001, IEEE Trans. Autom. Control..

[29]  van Stef Stef Eijndhoven,et al.  Linear quadratic regulator problem with positive controls , 1997, 1997 European Control Conference (ECC).

[30]  B. Anderson,et al.  Optimal control: linear quadratic methods , 1990 .

[31]  Xun Yu Zhou,et al.  Well-posedness and attainability of indefinite stochastic linear quadratic control in infinite time horizon , 2000 .

[33]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[34]  Xun Yu Zhou,et al.  Constrained Stochastic LQ Control with Random Coefficients, and Application to Portfolio Selection , 2005, SIAM J. Control. Optim..

[35]  R. Brammer,et al.  Differential controllability and the solution of linear inequalities--Part I , 1975 .

[36]  M. Pachter,et al.  The linear-quadratic optimal control problem with positive controllers† , 1980 .

[37]  Anton A. Stoorvogel,et al.  Linear quadratic regulator problem with positive controls , 1998 .

[38]  M. Fragoso,et al.  A New Approach to Linearly Perturbed Riccati Equations Arising in Stochastic Control , 1998 .

[39]  W. Wonham.  On a Matrix Riccati Equation of Stochastic Control , 1968 .