Convergence of an Upwind Finite-Difference Scheme for Hamilton–Jacobi–Bellman Equation in Optimal Control

This technical note considers convergence of an upwind finite-difference numerical scheme for the Hamilton-Jacobi-Bellman equation arising in optimal control. This effective scheme has been well-adapted and successfully applied to many examples. Nevertheless, its convergence has remained open until now. In this note, we show that the solution from this finite-difference scheme converges to the value function of the associated optimal control problem.

[1]  R. Newcomb.  VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS , 2010 .

[2]  Piero Lanucara,et al.  A splitting algorithm for Hamilton-Jacobi-Bellman equations , 1992 .

[3]  Kok Lay Teo,et al.  An upwind finite-difference method for the approximation of viscosity solutions to Hamilton-Jacobi-Bellman equations , 2000 .

[4]  William M. McEneaney,et al.  Convergence Rate for a Curse-of-dimensionality-Free Method for Hamilton--Jacobi--Bellman PDEs Represented as Maxima of Quadratic Forms , 2009, SIAM J. Control. Optim..

[5]  G. Lantoine,et al.  Quadratically Constrained Linear-Quadratic Regulator Approach for Finite-Thrust Orbital Rendezvous , 2012 .

[6]  M. Crandall.  Viscosity solutions: A primer , 1997 .

[7]  P. Souganidis.  Approximation schemes for viscosity solutions of Hamilton-Jacobi equations , 1985 .

[8]  P. Lions,et al.  Two approximations of solutions of Hamilton-Jacobi equations , 1984 .

[9]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[10]  H.-H. Wang,et al.  Successive approximation approach of optimal control for nonlinear discrete-time systems , 2005, Int. J. Syst. Sci..

[11]  Joel W. Burdick,et al.  Linear Hamilton Jacobi Bellman Equations in high dimensions , 2014, 53rd IEEE Conference on Decision and Control.

[12]  Iain Smears,et al.  On the Convergence of Finite Element Methods for Hamilton-Jacobi-Bellman Equations , 2011, SIAM J. Numer. Anal..

[13]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[14]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1990, 29th IEEE Conference on Decision and Control.

[15]  John C. Strikwerda,et al.  Finite Difference Methods for the Stokes and Navier–Stokes Equations , 1984 .

[16]  Hasnaa Zidani,et al.  Convergence of a non-monotone scheme for Hamilton–Jacobi–Bellman equations with discontinous initial data , 2010, Numerische Mathematik.

[17]  Hamilton-Jacobi-Bellman equations..  Approximation of optimal controls for semilinear parabolic PDE by solving , .

[18]  Karl Kunisch,et al.  POD-based feedback control of the burgers equation by solving the evolutionary HJB equation , 2005 .

[19]  William M. McEneaney,et al.  Max-plus methods for nonlinear control and estimation , 2005 .

[20]  Bao-Zhu Guo,et al.  Approximation of optimal feedback control: a dynamic programming approach , 2010, J. Glob. Optim..