Numerical Methods for Pursuit-Evasion Games via Viscosity Solutions

We present a class of numerical schemes for the Isaacs equation of pursuit-evasion games. We consider continuous value functions, where the solution is interpreted in the viscosity sense, as well as discontinuous value functions, where the notion of viscosity envelope-solution is needed. The convergence of the approximation scheme to the value function of the game is proved in both cases. A priori estimates of the convergence in L∞ are established when the value function is Holder continuous. We also treat problems with state constraints and discuss several issues concerning the implementation of the approximation scheme, the synthesis of approximate feedback controls, and the approximation of optimal trajectories. The efficiency of the algorithm is illustrated by a number of numerical tests, either in the case of one player (i.e., minimum time problem) or for some 2-players games.

[1]  W. Fleming.  The convergence problem for differential games , 1961 .

[2]  Rufus Isaacs,et al.  Differential Games , 1965 .

[3]  A. Friedman.  Differential games , 1971 .

[4]  J. Warga.  Optimal control of differential and functional equations , 1972 .

[5]  R. Elliott,et al.  The Existence Of Value In Differential Games , 1972 .

[6]  R. Elliott,et al.  Cauchy problems for certain Isaacs-Bellman equations and games of survival , 1974 .

[7]  P. Lions,et al.  Viscosity solutions of Hamilton-Jacobi equations , 1983 .

[8]  P. Lions.  Optimal control of diffusion processes and hamilton–jacobi–bellman equations part 2 : viscosity solutions and uniqueness , 1983 .

[9]  P. L. Linos.  Optimal control of diffustion processes and hamilton-jacobi-bellman equations part I: the dynamic programming principle and application , 1983 .

[10]  I. Dolcetta.  On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming , 1983 .

[11]  P. Lions.  Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations, Part I , 1983 .

[12]  P. Souganidis,et al.  Differential Games and Representation Formulas for Solutions of Hamilton-Jacobi-Isaacs Equations. , 1983 .

[13]  L. Evans,et al.  Differential games and nonlinear first order PDE on bounded domains , 1984 .

[14]  P. Lions,et al.  Some Properties of Viscosity Solutions of Hamilton-Jacobi Equations. , 1984 .

[15]  L. Evans,et al.  Viscosity solutions of Isaacs' equations and differential games with Lipschitz controls , 1984 .

[16]  P. Lions,et al.  Two approximations of solutions of Hamilton-Jacobi equations , 1984 .

[17]  H. Ishii,et al.  Approximate solutions of the bellman equation of deterministic control theory , 1984 .

[18]  Tai-Ping Liu,et al.  On a nonstrictly hyperbolic system of conservation laws , 1985 .

[19]  P. Souganidis.  Max-min representations and product formulas for the viscosity solutions of Hamilton-Jacobi equations with applications to differential games , 1985 .

[20]  P. Souganidis.  Approximation schemes for viscosity solutions of Hamilton-Jacobi equations , 1985 .

[21]  H. Ishii.  Perron’s method for Hamilton-Jacobi equations , 1987 .

[22]  G. Barles,et al.  Discontinuous solutions of deterministic optimal stopping time problems , 1987 .

[23]  M. Falcone.  A numerical approach to the infinite horizon problem of deterministic control theory , 1987 .

[24]  A. I. Subbotin,et al.  Semicontinuous solutions of Hamilton-Jacobi equations☆ , 1988 .

[25]  G. Barles,et al.  Exit Time Problems in Optimal Control and Vanishing Viscosity Method , 1988 .

[26]  Hitoshi Ishii,et al.  A boundary value problem of the Dirichlet type for Hamilton-Jacobi equations , 1989 .

[27]  M. Bardi,et al.  A PDE framework for games of pursuit-evasion type , 1989 .

[28]  E. Barron,et al.  Semicontinuous Viscosity Solutions For Hamilton–Jacobi Equations With Convex Hamiltonians , 1990 .

[29]  M. Falcone,et al.  An approximation scheme for the minimum time function , 1990 .

[30]  M. Bardi,et al.  Hamilton-Jacobi equations with singular boundary conditions on a free boundary and applications to differential games , 1991 .

[31]  G. Barles,et al.  Convergence of approximation schemes for fully nonlinear second order equations , 1991 .

[32]  B. Alziary de Roquefort.  Jeux différentiels et approximation numérique de fonctions valeur. 1re partie : étude théorique , 1991 .

[33]  M. Bardi,et al.  Approximation of differential games of pursuit-evasion by discrete-time games , 1991 .

[34]  Pierpaolo Soravia.  The concept of value in differential games of survival and viscosity solutions of Hamilton-Jacobi equations , 1992 .

[35]  Elisabeth Rouy,et al.  NUMERICAL APPROXIMATION OF VISCOSITY SOLUTIONS OF FIRST-ORDER HAMILTON-JACOBI EQUATIONS WITH NEUMANN TYPE BOUNDARY CONDITIONS , 1992 .

[36]  Piero Lanucara,et al.  A splitting algorithm for Hamilton-Jacobi-Bellman equations , 1992 .

[37]  Hölder continuity of the minimum-time function forC1-manifold targets , 1992 .

[38]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[39]  R. L. V. Gonzalez,et al.  Sur l'ordre de convergence des solutions discrétisées en temps et en espace de l'équation de Hamilton-Jacobi , 1992 .

[40]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[41]  A. I. Subbotin,et al.  Discontinuous solutions of a Dirichlet-type boundary-value problem for the first-order partial differential equation , 1993 .

[42]  H. J. Pesch,et al.  Complex differential games of pursuit-evasion type with state constraints, part 1: Necessary conditions for optimal open-loop strategies , 1993 .

[43]  H. Frankowska.  Lower semicontinuous solutions of Hamilton-Jacobi-Bellman equations , 1993 .

[44]  G. Barles.  Discontinuous viscosity solutions of first-order Hamilton-Jacobi equations: a guided visit , 1993 .

[45]  Pierpaolo Soravia.  Pursuit-evasion problems and viscosity solutions of Isaacs equations , 1993 .

[46]  Pierpaolo Soravia,et al.  Discontinuous viscosity solutions to dirichlet problems for hamilton-jacob1 equations with , 1993 .

[47]  M. Bardi,et al.  The Bellman equation for time-optimal control of noncontrollable, nonlinear systems , 1993 .

[48]  Matthew R. James,et al.  A partial differential inequality for dissipative nonlinear systems , 1993 .

[49]  G. Barles.  Solutions de viscosité des équations de Hamilton-Jacobi , 1994 .

[50]  Pierpaolo Soravia.  Generalized motion of a front propagating along its normal direction: a differential games approach , 1994 .

[51]  Alexander M. Tarasyev,et al.  Approximation schemes for constructing minimax solutions of Hamilton-Jacobi equations , 1994 .

[52]  H. J. Pesch.  Solving optimal control and pursuit-evasion game problems of high complexity , 1994 .

[53]  Marizio Falcone,et al.  Discrete time high-order schemes for viscosity solutions of Hamilton-Jacobi-Bellman equations , 1994 .

[54]  Joseph Lewin,et al.  Differential Games , 1994 .

[55]  L. Berkovitz.  A Theory of Differential Games , 1994 .

[56]  Maurizio Falcone.  The minimum time problem and its applications to front propagation , 1994 .

[57]  Jinchao Xu,et al.  Domain Decomposition Methods in Scientific and Engineering Computing , 1994 .

[58]  M. Falcone,et al.  Fully Discrete Schemes for the Value Function of Pursuit-Evasion Games , 1994 .

[59]  T. Başar,et al.  Advances in Dynamic Games and Applications , 1994 .

[60]  H. J. Pesch.  A Practical Guide to the Solution of Real-Life Optimal Control Problems , 1994 .

[61]  Pierre-Louis Lions,et al.  A GRID REFINEMENT METHOD FOR DETERMINISTIC CONTROL AND DIFFERENTIAL GAMES , 1994 .

[62]  H. J. Pesch,et al.  Three-Dimensional Air Combat: Numerical Solution of Complex Differential Games , 1995 .

[63]  M. Tidball.  Undiscounted zero sum differential games with stopping times , 1995 .

[64]  On the state constraint problem for differential games , 1995 .

[65]  Matthew R. James,et al.  Numerical approximation of the H∞ norm for nonlinear systems , 1995, Autom..

[66]  G. Olsder.  New trends in dynamic games and applications , 1995 .

[67]  Une procédure numérique pour la minimisation du coût maximum , 1995 .

[68]  W. Fleming,et al.  Risk-Sensitive Control on an Infinite Time Horizon , 1995 .

[69]  Andrei I. Subbotin,et al.  Generalized solutions of first-order PDEs - the dynamical optimization perspective , 1994, Systems and control.

[70]  M. Falcone,et al.  Convergence of Discrete Schemes for Discontinuous Value Functions of Pursuit-Evasion Games , 1995 .

[71]  Fabio Camilli,et al.  Computation of the H ∞ norm for nonlinear systems: a convergence result , 1996 .

[72]  F. Camilli,et al.  Approximation of Optimal Control Problems with State Constraints: Estimates and Applications , 1996 .

[73]  H. Ishii,et al.  A New Formulation of State Constraint Problems for First-Order PDEs , 1996 .

[74]  Pierpaolo Soravia.  ${\cal H}_\infty$ Control of Nonlinear Systems: Differential Games and Viscosity Solutions , 1996 .

[75]  L. Grüne.  An adaptive grid scheme for the discrete Hamilton-Jacobi-Bellman equation , 1997 .

[76]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[77]  A. Tarasyev.  Optimal Control Synthesis in Grid Approximation Schemes , 1997 .

[78]  Pierpaolo Soravia,et al.  On Differential Games for Infinite-Dimensional Systems with Nonlinear, Unbounded Operators , 1997 .

[79]  Pierpaolo Soravia,et al.  Estimates of Convergence of Fully Discrete Schemes for the Isaacs Equation of Pursuit-Evasion Differential Games Via Maximum Principle , 1998 .

[80]  P. Saint-Pierre,et al.  Set-Valued Numerical Analysis for Optimal Control and Differential Games , 1999 .

[81]  P. Souganidis.  Two-Player, Zero-Sum Differential Games and Viscosity Solutions , 1999 .

[82]  Pierpaolo Soravia.  Optimality principles and representation formulas for viscosity solutions of Hamilton-Jacobi equations. I. Equations of unbounded and degenerate control problems without uniqueness , 1999 .

[83]  M. Bardi,et al.  Pursuit-evasion games with state constraints: dynamic programming and discrete-time approximations , 2000 .

[84]  Pierpaolo Soravia,et al.  Differential Games and Nonlinear \boldmath$\cal HINFINITY$ Control in Infinite Dimensions , 2000, SIAM J. Control. Optim..