A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation

In this paper, we present a feedback design for numerical solution to optimal control problems, which is based on solving the corresponding Hamilton-Jacobi-Bellman (HJB) equation. An upwind finite-difference scheme is adopted to solve the HJB equation under the framework of the dynamic programming viscosity solution (DPVS) approach. Different from the usual existing algorithms, the numerical control function is interpolated in turn to gain the approximation of optimal feedback control-trajectory pair. Five simulations are executed and both of them, without exception, output the accurate numerical results. The design can avoid solving the HJB equation repeatedly, thus efficaciously promote the computation efficiency and save memory.

[1]  Karl Kunisch,et al.  Polynomial Approximation of High-Dimensional Hamilton-Jacobi-Bellman Equations and Applications to Feedback Control of Semilinear Parabolic PDEs , 2017, SIAM J. Sci. Comput..

[2]  P. Lions,et al.  Viscosity solutions of Hamilton-Jacobi equations , 1983 .

[3]  Bing Sun,et al.  Numerical solution to the optimal feedback control of continuous casting process , 2007, J. Glob. Optim..

[4]  X. Zhou.  Verification Theorems within the Framework of Viscosity Solutions , 1993 .

[5]  Kok Lay Teo,et al.  An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations , 2000 .

[6]  J. Stoer,et al.  Introduction to Numerical Analysis , 2002 .

[7]  Bao-Zhu Guo,et al.  Convergence of an Upwind Finite-Difference Scheme for Hamilton–Jacobi–Bellman Equation in Optimal Control , 2015, IEEE Transactions on Automatic Control.

[8]  Karl Kunisch,et al.  POD-based feedback control of the burgers equation by solving the evolutionary HJB equation , 2005 .

[9]  Delfim F. M. Torres,et al.  A TB-HIV/AIDS coinfection model and optimal control treatment , 2015, 1501.03322.

[10]  M. Falcone,et al.  Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations , 2014 .

[12]  M. Crandall.  Viscosity solutions: A primer , 1997 .

[13]  Lei Xie,et al.  HJB-POD-Based Feedback Design for the Optimal Control of Evolution Problems , 2004, SIAM J. Appl. Dyn. Syst..

[14]  P. Lions,et al.  Some Properties of Viscosity Solutions of Hamilton-Jacobi Equations. , 1984 .

[15]  Bing Sun,et al.  Dynamic programming approach to the numerical solution of optimal control with paradigm by a mathematical model for drug therapies of HIV/AIDS , 2014 .

[16]  M. Falcone,et al.  Coupling MPC and HJB for the Computation of POD-Based Feedback Laws , 2017, Lecture Notes in Computational Science and Engineering.

[17]  Bernard Haasdonk,et al.  Feedback control of parametrized PDEs via model order reduction and dynamic programming principle , 2018, Advances in Computational Mathematics.

[18]  K. Kunisch,et al.  12. Suboptimal Feedback Control of Flow Separation by POD Model Reduction , 2007 .

[19]  Yu-Chi Ho.  On centralized optimal control , 2005, IEEE Transactions on Automatic Control.

[20]  William M. McEneaney,et al.  Convergence Rate for a Curse-of-dimensionality-Free Method for Hamilton--Jacobi--Bellman PDEs Represented as Maxima of Quadratic Forms , 2009, SIAM J. Control. Optim..

[21]  K. Kunisch,et al.  Feedback stabilization of the three-dimensional Navier-Stokes equations using generalized Lyapunov equations , 2019, 1909.13757.