Multilevel Techniques for the Solution of HJB Minimum-Time Control Problems

The approximation of feedback control via the Dynamic Programming approach is a challenging problem. The computation of the feedback requires the knowledge of the value function, which can be characterized as the unique viscosity solution of a nonlinear Hamilton-Jacobi-Bellman (HJB) equation. The major obstacle is that the numerical methods known in literature strongly suffer when the dimension of the discretized problem becomes large. This is a strong limitation to the application of classical numerical schemes for the solution of the HJB equation in real applications. To tackle this problem, a new multi-level numerical framework is proposed. Numerical evidences show that classical methods have good smoothing properties, which allow one to use them as smoothers in a multilevel strategy. Moreover, a new smoother iterative scheme based on the Anderson acceleration of the classical value function iteration is introduced. The effectiveness of our new framework is proved by several numerical experiments focusing on minimum-time control problems.

[1]  M. Pollatschek,et al.  Algorithms for Stochastic Games with Geometrical Interpretation , 1969 .

[2]  David L. Elliott,et al.  Geometric control theory , 2000, IEEE Trans. Autom. Control..

[3]  Jan C. Willems,et al.  300 years of optimal control: From the brachystochrone to the maximum principle , 1997 .

[4]  S. Volkwein,et al.  Proper Orthogonal Decomposition for Linear-Quadratic Optimal Control , 2013 .

[5]  Yanping Chen,et al.  Variational discretization for parabolic optimal control problems with control constraints , 2012, J. Syst. Sci. Complex..

[6]  Raul Tempone,et al.  On the Connection between the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks , 2014 .

[7]  Chunjiang Qian,et al.  State feedback stabilization for a class of nonlinear time-delay systems via dynamic linear controllers , 2014, J. Syst. Sci. Complex..

[8]  Martin L. Puterman,et al.  On the Convergence of Policy Iteration in Stationary Dynamic Programming , 1979, Math. Oper. Res..

[9]  Michael Ulbrich,et al.  Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces , 2011, MOS-SIAM Series on Optimization.

[10]  Homer F. Walker,et al.  Anderson Acceleration for Fixed-Point Iterations , 2011, SIAM J. Numer. Anal..

[11]  Adriano Festa,et al.  Reconstruction of Independent Sub-domains for a class of Hamilton Jacobi Equations and its Application to Parallel Computing , 2014, 1405.3521.

[12]  Marianne Akian,et al.  Multigrid methods for two‐player zero‐sum stochastic games , 2011, Numer. Linear Algebra Appl..

[13]  Bing Sun,et al.  Dynamic Programming Viscosity Solution Approach and Its Applications to Optimal Control Problems , 2019, Studies in Systems, Decision and Control.

[14]  C. T. Kelley,et al.  Convergence Analysis for Anderson Acceleration , 2015, SIAM J. Numer. Anal..

[15]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[16]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[17]  Martin J. Gander,et al.  Scientific Computing - An Introduction using Maple and MATLAB , 2014 .

[18]  H. Hermes,et al.  Foundations of optimal control theory , 1968 .

[19]  Ronald A. Howard,et al.  Dynamic Programming , 1966 .

[20]  M. Falcone,et al.  Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations , 2014 .

[21]  M. Falcone,et al.  An approximation scheme for the minimum time function , 1990 .

[22]  Donald G. M. Anderson.  Iterative Procedures for Nonlinear Integral Equations , 1965, JACM.

[23]  John Rust,et al.  Convergence Properties of Policy Iteration , 2003, SIAM J. Control. Optim..

[24]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[25]  D. Sorensen.  Numerical methods for large eigenvalue problems , 2002, Acta Numerica.

[26]  Maurizio Falcone,et al.  A Patchy Dynamic Programming Scheme for a Class of Hamilton-Jacobi-Bellman Equations , 2011, SIAM J. Sci. Comput..

[27]  Nianyu Yi,et al.  Variational discretization for optimal control problems governed by parabolic equations , 2013, J. Syst. Sci. Complex..

[28]  Alfio Borzì,et al.  Formulation and Numerical Solution of Quantum Control Problems , 2017 .

[29]  Alessandro Alla,et al.  An Efficient Policy Iteration Algorithm for Dynamic Programming Equations , 2013, SIAM J. Sci. Comput..

[30]  Bao-Zhu Guo,et al.  Numerical solution to optimal feedback control by dynamic programming approach: A local approximation algorithm , 2017, J. Syst. Sci. Complex..

[31]  Long Chen.  INTRODUCTION TO MULTIGRID METHODS , 2005 .

[32]  Bernard Haasdonk,et al.  Feedback control of parametrized PDEs via model order reduction and dynamic programming principle , 2018, Advances in Computational Mathematics.

[33]  Justin W. L. Wan,et al.  Multigrid Methods for Second Order Hamilton-Jacobi-Bellman and Hamilton-Jacobi-Bellman-Isaacs Equations , 2013, SIAM J. Sci. Comput..