Solving Stochastic LQR Problems by Polynomial Chaos

We consider the infinite dimensional stochastic linear quadratic optimal control problem for the infinite horizon case. We provide a numerical framework for solving this problem using a polynomial chaos expansion approach. By applying the method of chaos expansions to the state equation, we obtain a system of deterministic partial differential equations in terms of the coefficients of the state and the control variables. We set up a control problem for each equation, which results in a set of infinite horizon deterministic linear quadratic regulator problems. We prove the optimality of the solution expressed in terms of the expansion of these coefficients compared to the direct approach. We perform numerical experiments which validate our approach and compare the finite and infinite horizon case.

[1]  J. Zabczyk,et al.  Stochastic Equations in Infinite Dimensions , 2008 .

[2]  Hermann Mena,et al.  On the benefits of the LDLT factorization for large-scale differential matrix equation solvers , 2015 .

[3]  Thorsten Gerber,et al.  Semigroups Of Linear Operators And Applications To Partial Differential Equations , 2016 .

[4]  Thilo Penzl.  LYAPACK A MATLAB Toolbox for Large Lyapunov and Riccati Equations , Model Reduction Problems , and Linear – Quadratic Optimal Control Problems Users , 2000 .

[5]  Amjad Tuffaha,et al.  A Numerical Approximation Framework for the Stochastic Linear Quadratic Regulator on Hilbert Spaces , 2017 .

[6]  Tony Stillfjord,et al.  Low-Rank Second-Order Splitting of Large-Scale Differential Riccati Equations , 2015, IEEE Transactions on Automatic Control.

[7]  Peter Benner,et al.  Low rank methods for a class of generalized Lyapunov equations and related issues , 2013, Numerische Mathematik.

[8]  Amjad Tuffaha,et al.  The Stochastic Linear Quadratic Control Problem with Singular Estimates , 2017, SIAM J. Control. Optim..

[9]  V. Mehrmann,et al.  Convergence of the backward Euler scheme for the operator-valued Riccati differential equation with semi-definite data , 2018, Evolution Equations & Control Theory.

[10]  Veit Hagenmeyer,et al.  Comments on Truncation Errors for Polynomial Chaos Expansions , 2017, IEEE Control Systems Letters.

[11]  Mark R. Opmeer,et al.  Finite-Rank ADI Iteration for Operator Lyapunov Equations , 2013, SIAM J. Control. Optim..

[12]  Gianmario Tessitore,et al.  Backward Stochastic Riccati Equations and Infinite Horizon L-Q Optimal Control with Infinite Dimensional State Space and Random Coefficients , 2008 .

[13]  Peter Benner,et al.  Dual Pairs of Generalized Lyapunov Inequalities and Balanced Truncation of Stochastic Linear Systems , 2015, IEEE Transactions on Automatic Control.

[14]  S. Sager,et al.  Solving Stochastic Optimal Control Problems by a Wiener Chaos Approach , 2014, Vietnam Journal of Mathematics.

[15]  G. Da Prato.  Direct solution of a riccati equation arising in stochastic control theory , 1984 .

[16]  Peter Benner,et al.  Rosenbrock Methods for Solving Riccati Differential Equations , 2013, IEEE Transactions on Automatic Control.

[17]  David L. Russel.  Representation and Control of Infinite Dimensional Systems, Vols. 1 and 2 (A. Bensonssan, G. Da Prato, M. Delfour, and S. Mitter) , 1995, SIAM Rev..

[18]  H. Mena,et al.  The stochastic linear quadratic optimal control problem in Hilbert spaces: A polynomial chaos approach , 2016 .

[19]  Hermann Mena,et al.  Numerical solution of the finite horizon stochastic linear quadratic control problem , 2017, Numer. Linear Algebra Appl..

[20]  Akira Ichikawa,et al.  Quadratic control for linear time-varying systems , 1990 .

[21]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[22]  John A. Burns,et al.  Mesh Independence of Kleinman--Newton Iterations for Riccati Equations in Hilbert Space , 2008, SIAM J. Control. Optim..

[23]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[24]  Gianmario Tessitore,et al.  On the Backward Stochastic Riccati Equation in Infinite Dimensions , 2005, SIAM J. Control. Optim..

[25]  B. Rozovskii,et al.  Wiener chaos solutions of linear stochastic evolution equations , 2005, math/0504558.