Numerical Methods for differential Games Based on Partial differential equations

In this paper we present some numerical methods for the solution of two-persons zero-sum deterministic differential games. The methods are based on the dynamic programming approach. We first solve the Isaacs equation associated to the game to get an approximate value function and then we use it to reconstruct approximate optimal feedback controls and optimal trajectories. The approximation schemes also have an interesting control interpretation since the time-discrete scheme stems from a dynamic programming principle for the associated discrete time dynamical system. The general framework for convergence results to the value function is the theory of viscosity solutions. Numerical experiments are presented solving some classical pursuit-evasion games.

[1]  M. Bardi,et al.  Approximation of differential games of pursuit-evasion by discrete-time games , 1991 .

[2]  P. Souganidis,et al.  Differential Games and Representation Formulas for Solutions of Hamilton-Jacobi-Isaacs Equations. , 1983 .

[3]  P. Saint-Pierre,et al.  Set-Valued Numerical Analysis for Optimal Control and Differential Games , 1999 .

[4]  Mabel M. Tidball,et al.  Fast solution of general nonlinear fixed point problems , 1992 .

[5]  Pierpaolo Soravia.  Pursuit-evasion problems and viscosity solutions of Isaacs equations , 1993 .

[6]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[7]  M. G. Delgado,et al.  Optimal control and partial differential equations , 2004 .

[8]  Maurizio Falcone,et al.  Advances in Parallel Algorithms for the Isaacs Equation , 2005 .

[9]  Phil Howlett,et al.  Stochastic Optimal Control of a Solar Car , 2001 .

[10]  A. I. Subbotin.  Existence and uniqueness results for Hamilton-Jacobi equations , 1991 .

[11]  Antony W Merz,et al.  The Homicidal Chauffeur - A Differential Game , 1971 .

[12]  L. Evans,et al.  Differential games and nonlinear first order PDE on bounded domains , 1984 .

[13]  M. Falcone,et al.  Fully Discrete Schemes for the Value Function of Pursuit-Evasion Games , 1994 .

[14]  P. Varaiya,et al.  Differential games , 1971 .

[15]  I. Dolcetta.  On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming , 1983 .

[16]  Chi-Tien Lin,et al.  $L^1$-Stability and error estimates for approximate Hamilton-Jacobi solutions , 2001, Numerische Mathematik.

[17]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1991 .

[18]  M. Falcone,et al.  Convergence of Discrete Schemes for Discontinuous Value Functions of Pursuit-Evasion Games , 1995 .

[19]  T. Başar,et al.  Stochastic and differential games : theory and numerical methods , 1999 .

[20]  P. Lions,et al.  Viscosity solutions of Hamilton-Jacobi equations , 1983 .

[21]  Hitoshi Ishii,et al.  A boundary value problem of the Dirichlet type for Hamilton-Jacobi equations , 1989 .

[22]  H. Frankowska.  Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations , 1993 .

[23]  Michael H. Breitner,et al.  Real-Time Computation of Strategies of Differential Games with Applications to Collision Avoidance , 1998 .

[24]  Jean-Pierre Aubin,et al.  Viability theory , 1991 .

[25]  Alexander M. Tarasyev.  Control synthesis in grid schemesfor Hamilton‐Jacobi equations , 1999, Ann. Oper. Res..

[26]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[27]  P. Souganidis,et al.  Differential Games, Optimal Control and Directional Derivatives of Viscosity Solutions of Bellman’s and Isaacs’ Equations , 1985 .

[28]  Semyon Tsynkov,et al.  Finite-Difference Schemes for Partial Differential Equations , 2006 .

[29]  R. Elliott,et al.  The Existence Of Value In Differential Games , 1972 .

[30]  B. Alziary de Roquefort.  Jeux différentiels et approximation numérique de fonctions valeur. 1re partie : étude théorique , 1991 .

[31]  R. Bellman.  Dynamic programming. , 1957, Science.

[32]  M. K rn,et al.  Stochastic Optimal Control , 1988 .

[33]  E. Barron.  Differential games maximum cost , 1990 .

[34]  Barry Smith,et al.  Domain Decomposition Methods for Partial Differential Equations , 1997 .

[35]  H. J. Pesch,et al.  Three-Dimensional Air Combat: Numerical Solution of Complex Differential Games , 1995 .

[36]  P. Lions,et al.  Two approximations of solutions of Hamilton-Jacobi equations , 1984 .

[37]  L. Evans,et al.  Viscosity solutions of Isaacs' equations and differential games with Lipschitz controls , 1984 .

[38]  P. Lions,et al.  Some Properties of Viscosity Solutions of Hamilton-Jacobi Equations. , 1984 .

[39]  M. Falcone,et al.  Discrete approximation of the minimal time function for systems with regular optimal trajectories , 1990 .

[40]  Pierpaolo Soravia,et al.  Discontinuous viscosity solutions to dirichlet problems for hamilton-jacob1 equations with , 1993 .

[41]  M. Bardi,et al.  Pursuit-evasion games with state constraints: dynamic programming and discrete-time approximations , 2000 .

[42]  Pierre-Louis Lions,et al.  A GRID REFINEMENT METHOD FOR DETERMINISTIC CONTROL AND DIFFERENTIAL GAMES , 1994 .

[43]  Pierpaolo Soravia,et al.  Estimates of Convergence of Fully Discrete Schemes for the Isaacs Equation of Pursuit-Evasion Differential Games Via Maximum Principle , 1998 .

[44]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[45]  E. Barron,et al.  OPTIMAL CONTROL AND SEMICONTINUOUS VISCOSITY SOLUTIONS , 1991 .

[46]  Martino Bardi,et al.  A boundary value problem for the minimum-time function , 1989 .

[47]  M. Bardi,et al.  Hamilton-Jacobi equations with singular boundary conditions on a free boundary and applications to differential games , 1991 .

[48]  A. I. Subbotin,et al.  Discontinuous solutions of a Dirichlet-type boundary-value problem for the first-order partial differential equation , 1993 .

[49]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1990, 29th IEEE Conference on Decision and Control.

[50]  J. Henry,et al.  System Modelling and Optimization: Proceedings of the 16th IFIP-TC7 Conference, Compiègne, France, July 5-9, 1993 , 1994, System Modelling and Optimization.

[51]  G. Barles.  Solutions de viscosité des équations de Hamilton-Jacobi , 1994 .

[52]  Piero Lanucara,et al.  Parallel Algorithms for the Isaacs Equation , 2001 .

[53]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[54]  J. L. Lions,et al.  Analysis and Optimization of Systes , 1990 .

[55]  P. Saint-Pierre,et al.  Numerical Schemes for Discontinuous Value Functions of Optimal Control , 2000 .

[56]  Martin L. Puterman,et al.  On the Convergence of Policy Iteration in Stationary Dynamic Programming , 1979, Math. Oper. Res..

[57]  Roberto Ferretti,et al.  Convergence of Semi-Lagrangian Approximations to Convex Hamilton-Jacobi Equations under (Very) Large Courant Numbers , 2002, SIAM J. Numer. Anal..

[58]  V. N. Ushakov,et al.  Approximation schemas and finite-difference operators for constructing generalized solutions of Hamilton-Jacobi equations , 1995 .

[59]  M. Falcone,et al.  An approximation scheme for the minimum time function , 1990 .

[60]  M. Bardi,et al.  A PDE framework for games of pursuit-evasion type , 1989 .

[61]  Andrei I. Subbotin,et al.  Generalized solutions of first-order PDEs - the dynamical optimization perspective , 1994, Systems and control.

[62]  Leonard D. Berkovitz,et al.  A survey of recent results in differential games , 1989 .

[63]  Pierpaolo Soravia.  The concept of value in differential games of survival and viscosity solutions of Hamilton-Jacobi equations , 1992 .

[64]  A. I. Subbotin.  Generalization of the main equation of differential game theory , 1984 .

[65]  E. Altman,et al.  Approximations In Dynamic Zero-Sum Games , 1994 .

[66]  M. Falcone,et al.  Numerical Methods for Pursuit-Evasion Games via Viscosity Solutions , 1999 .

[67]  Martino Bardi,et al.  Stochastic and Differential Games , 1999 .

[68]  Mabel M. Tidball,et al.  Zero Sum Differential Games With Stopping Times: Some Results About its Numerical Resolution , 1994 .

[69]  E. Altman,et al.  Approximations in Dynamic Zero-Sum Games II , 1997 .

[70]  W. Fleming,et al.  Risk sensitive optimal control and differential games , 1992 .

[71]  A. I. Subbotin.  Generalized Solutions of First Order PDEs , 1995 .

[72]  J. Filar,et al.  Advances in Dynamic Games and Applications , 2001 .

[73]  P. Souganidis,et al.  Differential games and directional derivatives of viscosity solutions of Issacs' equations II , 1986 .

[74]  H. J. Pesch.  Solving optimal control and pursuit-evasion game problems of high complexity , 1994 .

[75]  P. Lions.  Generalized Solutions of Hamilton-Jacobi Equations , 1982 .

[76]  G. Barles.  Discontinuous viscosity solutions of first-order Hamilton-Jacobi equations: a guided visit , 1993 .

[77]  G. Barles,et al.  Discontinuous solutions of deterministic optimal stopping time problems , 1987 .

[78]  P. Souganidis.  Max-min representations and product formulas for the viscosity solutions of Hamilton-Jacobi equations with applications to differential games , 1985 .