Efficient method for computing strategies for successive pursuit differential games

In successive pursuit, a pursuer seeks to capture as many evaders as possible in succession in the shortest amount of time. At the same time, a coalition of evaders seeks to maximize capture time (or prevent capture entirely) with or without the knowledge of the pursuer’s control law or preferred capture order. This study seeks to obtain a control strategy for both the pursuer and the coalition of evaders that is robust to uncertainty and variation in the pursuer or evader coalition strategy and that can be computed in a reasonable amount of time. A combination of techniques from differential game theory and discrete optimization are employed to compute such a strategy. In particular, a sub-optimal numerical approach using limited lookahead and a Monte Carlo tree search algorithm are used to obtain solutions in the presence of a high-dimensional action space. Examples are presented for both simple pursuit dynamics and the dynamics of the so-called Homicidal Chauffeur game. This work is sponsored by the Department of the Air Force under Air Force Contract FA8721-05-C-0002. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the United States Government.

[1]  A. I. Subbotin,et al.  Game-Theoretical Control Problems , 1987 .

[2]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[3]  V. Patsko,et al.  Homicidal Chauffeur Game: History and Modern Studies , 2011 .

[4]  Erik Blasch,et al.  Formation control in multi-player pursuit evasion game with superior evaders , 2007, SPIE Defense + Commercial Sensing.

[5]  R. Newcomb.  VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS , 2010 .

[6]  Yu. I. Berdyshev.  Choosing the sequence of approach of a nonlinear object to a group of moving points , 2011 .

[7]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[8]  Alexander Bolonkin,et al.  Geometry-Based Parametric Modeling for Single Pursuer Multiple Evader Problems , 2003 .

[9]  I. Shevchenko.  Strategies for Alternative Pursuit Games , 2009 .

[10]  R. Isaacs.  Differential games: a mathematical theory with applications to warfare and pursuit , 1999 .

[11]  Antonio Sartal,et al.  Solving the dynamic traveling salesman problem using a genetic algorithm with trajectory prediction: An application to fish aggregating devices , 2015, Comput. Oper. Res..

[12]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[13]  Yu. I. Berdyshev.  On a nonlinear problem of a sequential control with a parameter , 2008 .

[14]  E. P. Maslov,et al.  A Detection Evasion Problem , 2004 .

[15]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[16]  A. A. Chikrii,et al.  Pursuit of a group of evaders by a single controlled object , 1987 .

[17]  Nikolai D. Botkin,et al.  Stable Numerical Schemes for Solving Hamilton-Jacobi-Bellman-Isaacs Equations , 2011, SIAM J. Sci. Comput..

[18]  I. Shevchenko.  Minimizing the distance to one evader while chasing another , 2004 .

[19]  Simon M. Lucas,et al.  Monte-Carlo Tree Search for the Physical Travelling Salesman Problem , 2012, EvoApplications.

[20]  L. Evans,et al.  Viscosity solutions of Isaacs' equations and differential games with Lipschitz controls , 1984 .

[21]  Maarten P. D. Schadd,et al.  Addressing NP-Complete Puzzles with Monte-Carlo Methods 1 , 2008 .

[22]  Naira Hovakimyan,et al.  Some Sufficient Conditions for Multi-Player Pursuit-Evasion Games with Continuous and Discrete Observations , 2009 .

[23]  Simon M. Lucas,et al.  Rolling horizon evolution versus tree search for navigation in single-player real-time games , 2013, GECCO '13.

[24]  J. Doye,et al.  Global Optimization by Basin-Hopping and the Lowest Energy Structures of Lennard-Jones Clusters Containing up to 110 Atoms , 1997, cond-mat/9803344.

[25]  A. I. Subbotin.  Generalization of the main equation of differential game theory , 1984 .

[26]  Antony W Merz,et al.  The Homicidal Chauffeur - A Differential Game , 1971 .

[27]  P. Hagedorn,et al.  Point capture of two evaders in succession , 1979 .

[28]  A. G. Chentsov,et al.  Solving the dynamic traveling salesman game problem , 2010 .

[29]  Dongxu Li,et al.  Better cooperative control with limited look-ahead , 2006, 2006 American Control Conference.

[30]  William J. Cook,et al.  The Traveling Salesman Problem: A Computational Study , 2007 .

[31]  P. Souganidis,et al.  Differential Games and Representation Formulas for Solutions of Hamilton-Jacobi-Isaacs Equations. , 1983 .

[32]  Approaching Coalitions of Evaders on the Average , 2007 .

[33]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[34]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[35]  Naira Hovakimyan,et al.  Guaranteed Strategies for nonlinear Multi-Player Pursuit-Evasion Games , 2010, IGTR.

[36]  Dongxu Li,et al.  A Hierarchical Approach To Multi-Player Pursuit-Evasion Differential Games , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[37]  Peter I. Cowling,et al.  Monte Carlo Tree Search with macro-actions and heuristic route planning for the Physical Travelling Salesman Problem , 2012, 2012 IEEE Conference on Computational Intelligence and Games (CIG).

[38]  K. Zemskov,et al.  Construction of optimal position strategies in a differential pursuit-evasion game with one pursuer and two evaders☆ , 1997 .

[39]  I. Shevchenko.  Successive Pursuit with a Bounded Detection Domain , 1997 .

[40]  Genshe Chen,et al.  Multi-Pursuer Multi-Evader Pursuit-Evasion Games with Jamming Confrontation , 2007, J. Aerosp. Comput. Inf. Commun..

[41]  Maurizio Falcone,et al.  Numerical Methods for differential Games Based on Partial differential equations , 2006, IGTR.

[42]  Liang Tang,et al.  Hierarchical decomposition approach for pursuit-evasion differential game with multiple players , 2006, 2006 IEEE Aerospace Conference.

[43]  Dongxu Li,et al.  Multi-player pursuit-evasion differential games , 2006 .

[44]  Fabien Teytaud,et al.  Optimization of the Nested Monte-Carlo Algorithm on the Traveling Salesman Problem with Time Windows , 2011, EvoApplications.

[45]  C. Tomlin,et al.  Control Strategies in Multi-Player Pursuit and Evasion Game ⁄ , 2005 .

[46]  M. Falcone,et al.  Numerical Methods for Pursuit-Evasion Games via Viscosity Solutions , 1999 .

[47]  Varvara L. Turova,et al.  Level Sets of the Value Function in differential Games with the Homicidal Chauffeur Dynamics , 2001, IGTR.

[48]  Pramod P. Khargonekar,et al.  Cooperative defense within a single-pursuer, two-evader pursuit evasion differential game , 2010, 49th IEEE Conference on Decision and Control (CDC).

[49]  Geert Jan Olsder,et al.  Conic surveillance evasion , 1979 .

[50]  Pramod P. Khargonekar,et al.  Encouraging attacker retreat through defender cooperation , 2011, IEEE Conference on Decision and Control and European Control Conference.

[51]  Alexandre M. Bayen,et al.  A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games , 2005, IEEE Transactions on Automatic Control.

[52]  Dongxu Li,et al.  Improvement with Look-ahead on Cooperative Pursuit Games , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[53]  Sourabh Bhattacharya,et al.  Differential Game-Theoretic Approach to a Spatial Jamming Problem , 2013 .

[54]  Claire J. Tomlin,et al.  Collision Avoidance Strategies for a Three-Player Game , 2011 .

[55]  J. Lewin.  Differential Games: Theory And Methods For Solving Game Problems With Singular Surfaces , 2011 .

[56]  Zhengyuan Zhou,et al.  Evasion as a team against a faster pursuer , 2013, 2013 American Control Conference.