Polynomial Approximation of High-Dimensional Hamilton-Jacobi-Bellman Equations and Applications to Feedback Control of Semilinear Parabolic PDEs

A procedure for the numerical approximation of high-dimensional Hamilton--Jacobi--Bellman (HJB) equations associated to optimal feedback control problems for semilinear parabolic equations is proposed. Its main ingredients are a pseudospectral collocation approximation of the PDE dynamics and an iterative method for the nonlinear HJB equation associated to the feedback synthesis. The latter is known as the successive Galerkin approximation. It can also be interpreted as Newton iteration for the HJB equation. At every step, the associated linear generalized HJB equation is approximated via a separable polynomial approximation ansatz. Stabilizing feedback controls are obtained from solutions to the HJB equations for systems of dimension up to fourteen.

[1]  J. Burns,et al.  A control problem for Burgers' equation with bounded input/output , 1991 .

[2]  Elis Stefansson,et al.  Sequential alternating least squares for solving high dimensional linear Hamilton-Jacobi-Bellman equation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Randal W. Beard,et al.  Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation , 1997, Autom..

[4]  Nobuo Sannomiya,et al.  Suboptimal Feedback Control of a Nonlinear System , 1971 .

[5]  Maurizio Falcone,et al.  An Adaptive POD Approximation Method for the Control of Advection-Diffusion Equations , 2013, Control and Optimization with PDE Constraints.

[6]  J. Cloutier.  State-dependent Riccati equation techniques: an overview , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[7]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[8]  Bernie D. Shizgal,et al.  A pseudospectral method of solution of Fisher's equation , 2006 .

[9]  Richard Bellman,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[10]  Lars Grüne,et al.  Stabilization with discounted optimal control : the discrete time case , 2016 .

[11]  Jochen Garcke,et al.  Suboptimal Feedback Control of PDEs by Solving HJB Equations on Adaptive Sparse Grids , 2017, J. Sci. Comput..

[12]  Christoph Schwab,et al.  Sparse Tensor Galerkin Discretization of Parametric and Random Parabolic PDEs - Analytic Regularity and Generalized Polynomial Chaos Approximation , 2013, SIAM J. Math. Anal..

[13]  Vladimir Gaitsgory,et al.  Stabilization of strictly dissipative discrete time systems with discounted optimal control , 2018, Autom..

[14]  Albert Cohen,et al.  High-Dimensional Adaptive Sparse Polynomial Interpolation and Applications to Parametric PDEs , 2013, Foundations of Computational Mathematics.

[15]  Sérgio S. Rodrigues,et al.  Remarks on the Internal Exponential Stabilization to a Nonstationary Solution for 1D Burgers Equations , 2015, SIAM J. Control. Optim..

[16]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[17]  Dragan Nesic,et al.  Stability of infinite-horizon optimal control with discounted cost , 2014, 53rd IEEE Conference on Decision and Control.

[18]  Fredi Tröltzsch,et al.  Boundary feedback stabilization of the Schlögl system , 2015, Autom..

[19]  Max D. Gunzburger,et al.  Sparse Collocation Methods for Stochastic Interpolation and Quadrature , 2017 .

[20]  Claude Jeffrey Gittelson,et al.  Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs* , 2011, Acta Numerica.

[21]  Fabian R. Wirth,et al.  A Generalization of Zubov's Method to Perturbed Systems , 2001, SIAM J. Control. Optim..

[22]  Karl Kunisch,et al.  The linear regulator problem for parabolic systems , 1984 .

[23]  A. Quarteroni,et al.  Numerical Approximation of Partial Differential Equations , 2008 .

[24]  Harvey Thomas Banks,et al.  Feedback Control Methodologies for Nonlinear Systems , 2000 .

[25]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[26]  Lei Xie,et al.  HJB-POD-Based Feedback Design for the Optimal Control of Evolution Problems , 2004, SIAM J. Appl. Dyn. Syst..

[27]  R. Bellman.  A Markovian Decision Process , 1957 .

[28]  Wei Kang,et al.  Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations , 2015, Computational Optimization and Applications.

[29]  Massimo Fornasier,et al.  Un)conditional consensus emergence under perturbed and decentralized feedback controls , 2015 .

[30]  Dante Kalise,et al.  Reduced-order minimum time control of advection-reaction-diffusion systems via dynamic programming. , 2014 .

[31]  Alessandro Alla,et al.  An Efficient Policy Iteration Algorithm for Dynamic Programming Equations , 2013, SIAM J. Sci. Comput..

[32]  G. Saridis,et al.  Approximate Solutions to the Time-Invariant Hamilton–Jacobi–Bellman Equation , 1998 .

[33]  Roberto Ferretti.  Internal Approximation Schemes for Optimal Control Problems in Hilbert Spaces , 1997 .

[34]  Jean-Pierre Raymond,et al.  Nonlinear feedback stabilization of a two-dimensional Burgers equation , 2010 .

[35]  Karl Kunisch,et al.  Feedback stabilization of the Schlögl model by LQG-balanced truncation , 2015, 2015 European Control Conference (ECC).

[36]  M. Falcone,et al.  Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations , 2014 .

[37]  Michael Griebel,et al.  An Adaptive Sparse Grid Semi-Lagrangian Scheme for First Order Hamilton-Jacobi Bellman Equations , 2012, Journal of Scientific Computing.

[38]  Joel W. Burdick,et al.  Linear Hamilton Jacobi Bellman Equations in high dimensions , 2014, 53rd IEEE Conference on Decision and Control.

[39]  Sertac Karaman,et al.  High-dimensional stochastic optimal control using continuous tensor decompositions , 2016, Int. J. Robotics Res..