Finite-horizon covariance control for discrete-time stochastic linear systems subject to input constraints

Abstract This work deals with a finite-horizon covariance control problem for discrete-time, stochastic linear systems with complete state information subject to input constraints. First, we present the main steps for the transcription of the covariance control problem, which is originally formulated as a stochastic optimal control problem, into a deterministic nonlinear program (NLP) with a convex performance index and with both convex and non-convex constraints. In particular, the convex constraints in this nonlinear program are induced by the input constraints of the stochastic optimal control problem, whereas the non-convex constraints are induced by the requirement that the terminal state covariance be equal to a prescribed positive definite matrix. Subsequently, we associate this nonlinear program, via a simple convex relaxation technique, with a (convex) semi-definite program, which can be solved numerically by means of modern computational tools of convex optimization. Although, in general, the endpoints of a representative sample of closed-loop trajectories generated by the control policy that corresponds to the solution of the relaxed convex program are not expected to follow exactly the goal terminal Gaussian distribution, they are more likely to be concentrated near the mean of this distribution than if they were drawn from the latter, which is a desirable feature in practice. Numerical simulations that illustrate the key ideas of this work are also presented.

[1]  B. Jamison,et al.  Reciprocal Processes: The Stationary Gaussian Case , 1970 .

[2]  John Lygeros,et al.  On convexity of stochastic optimization problems with constraints , 2009, 2009 European Control Conference (ECC).

[3]  Robert Skelton,et al.  A covariance control theory , 1985, 1985 24th IEEE Conference on Decision and Control.

[4]  Tryphon T. Georgiou,et al.  Optimal Steering of a Linear Stochastic System to a Final Probability Distribution—Part III , 2014, IEEE Transactions on Automatic Control.

[5]  Robert E. Skelton,et al.  An improved covariance assignment theory for discrete systems , 1992 .

[6]  Basil Kouvaritakis,et al.  Model Predictive Control: Classical, Robust and Stochastic , 2015 .

[7]  Ian R. Petersen,et al.  State Distributions and Minimum Relative Entropy Noise Sequences in Uncertain Stochastic Systems: The Discrete-Time Case , 2012, SIAM J. Control. Optim..

[8]  Efstathios Bakolas,et al.  Stochastic linear systems subject to constraints , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[9]  John Lygeros,et al.  Stochastic receding horizon control with output feedback and bounded controls , 2012, Autom..

[10]  Karolos M. Grigoriadis,et al.  Covariance Controllers: A New Parameterization of the Class of All Stabilizing Controllers , 1990, 1990 American Control Conference.

[11]  Bernard C. Levy,et al.  Discrete-time Gauss-Markov processes with fixed reciprocal dynamics , 1997 .

[12]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[13]  Karolos M. Grigoriadis,et al.  Minimum-energy covariance controllers , 1997, Autom..

[14]  Stephen P. Boyd,et al.  Design of Affine Controllers via Convex Optimization , 2010, IEEE Transactions on Automatic Control.

[15]  John Lygeros,et al.  Stochastic Receding Horizon Control With Bounded Control Inputs: A Vector Space Approach , 2009, IEEE Transactions on Automatic Control.

[16]  A. Beghi,et al.  On the relative entropy of discrete-time Markov processes with given end-point densities , 1996, IEEE Trans. Inf. Theory.

[17]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[18]  Dimitri P. Bertsekas,et al.  Convex Optimization Algorithms , 2015 .

[19]  James A. Primbs,et al.  Stochastic Receding Horizon Control of Constrained Linear Systems With State and Control Multiplicative Noise , 2007, IEEE Transactions on Automatic Control.

[20]  Eric C. Kerrigan,et al.  Optimization over state feedback policies for robust control with constraints , 2006, Autom..