A Decentralized Fuzzy Learning Algorithm for Pursuit-Evasion Differential Games with Superior Evaders

In this paper, we consider multi-pursuer single-superior-evader pursuit-evasion differential games where the evader has a speed that is similar to or higher than the speed of each pursuer. A new fuzzy reinforcement learning algorithm is proposed in this work. The proposed algorithm uses the well-known Apollonius circle mechanism to define the capture region of the learning pursuer based on its location and the location of the superior evader. The proposed algorithm uses the Apollonius circle with a developed formation control approach in the tuning mechanism of the fuzzy logic controller (FLC) of the learning pursuer so that one or some of the learning pursuers can capture the superior evader. The formation control mechanism used by the proposed algorithm guarantees that the pursuers are distributed around the superior evader in order to avoid collision between pursuers. The formation control mechanism used by the proposed algorithm also makes the Apollonius circles of each two adjacent pursuers intersect or be at least tangent to each other so that the capture of the superior evader can occur. The proposed algorithm is a decentralized algorithm as no communication among the pursuers is required. The only information the proposed algorithm requires is the position and the speed of the superior evader. The proposed algorithm is used to learn different multi-pursuer single-superior-evader pursuit-evasion differential games. The simulation results show the effectiveness of the proposed algorithm.

[1]  Uzay Kaymak,et al.  Systems Control With Generalized Probabilistic Fuzzy-Reinforcement Learning , 2011, IEEE Transactions on Fuzzy Systems.

[2]  Rafael Murrieta-Cid,et al.  On the value of information in a differential pursuit-evasion game , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Emil M. Petriu,et al.  Mission-Driven Robotic Intelligent Sensor Agents for Territorial Security , 2011, IEEE Computational Intelligence Magazine.

[4]  Efstathios Bakolas,et al.  Evasion from a group of pursuers with double integrator kinematics , 2013, 52nd IEEE Conference on Decision and Control.

[5]  Howard M. Schwartz,et al.  Q(λ)‐learning adaptive fuzzy logic controllers for pursuit–evasion differential games , 2011 .

[6]  Tamer Basar,et al.  Numerical approximation for a visibility based pursuit-evasion game , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  L. Buşoniu,et al.  Fuzzy Partition Optimization for Approximate Fuzzy Q-iteration , 2008, IFAC Proceedings Volumes.

[8]  Andrea Bonarini,et al.  Reinforcement Learning in Continuous Action Spaces through Sequential Monte Carlo Methods , 2007, NIPS.

[9]  Wei Lin,et al.  Nash strategies for pursuit-evasion differential games involving limited observations , 2015, IEEE Transactions on Aerospace and Electronic Systems.

[10]  Leemon C. Baird,et al.  Residual Algorithms: Reinforcement Learning with Function Approximation , 1995, ICML.

[11]  Hong Bing-rong,et al.  Research on High Speed Evader vs. Multi Lower Speed Pursuers in Multi Pursuit-evasion Games , 2012 .

[12]  Eloy García,et al.  Active target defense differential game , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  Howard M. Schwartz,et al.  Self-learning fuzzy logic controllers for pursuit-evasion differential games , 2011, Robotics Auton. Syst..

[14]  Genshe Chen,et al.  A decentralized approach to pursuer-evader games with multiple superior evaders , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[15]  Howard M. Schwartz,et al.  The residual gradient FACL algorithm for differential games , 2015, 2015 IEEE 28th Canadian Conference on Electrical and Computer Engineering (CCECE).

[16]  Zhihua Qu,et al.  Pursuit-evasion games with multi-pursuer vs. one fast evader , 2010, 2010 8th World Congress on Intelligent Control and Automation.

[17]  Leslie Pack Kaelbling,et al.  Practical Reinforcement Learning in Continuous Spaces , 2000, ICML.

[18]  Amit Kumar,et al.  An evader-centric strategy against fast pursuer in an unknown environment with static obstacles , 2013, 2013 International Conference on Control, Automation, Robotics and Embedded Systems (CARE).

[19]  Ian Postlethwaite,et al.  A Cooperative Pursuit-Evasion Game for Non-holonomic Systems , 2014 .

[20]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[21]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[22]  Lionel Jouffe,et al.  Fuzzy inference system learning by reinforcement methods , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[23]  Jason M. O'Kane,et al.  A sampling-based algorithm for multi-robot visibility-based pursuit-evasion , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Renping Liu,et al.  A novel approach based on evolutionary game theoretic model for multi- player pursuit evasion , 2010, 2010 International Conference on Computer, Mechatronics, Control and Electronic Engineering.

[25]  Hugh F. Durrant-Whyte,et al.  A time-optimal control strategy for pursuit-evasion games problems , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[26]  Erik Blasch,et al.  Formation control in multi-player pursuit evasion game with superior evaders , 2007, SPIE Defense + Commercial Sensing.

[27]  Subhrajit Bhattacharya,et al.  Pursuit-evasion game for normal distributions , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Zhihua Qu,et al.  A heuristic task scheduling for multi-pursuer multi-evader games , 2011, 2011 IEEE International Conference on Information and Automation.

[29]  Mingyan Liu,et al.  Learning in Hide-and-Seek , 2014, IEEE/ACM Transactions on Networking.

[30]  Lining Sun,et al.  A novel hierarchical decomposition for multi-player pursuit evasion differential game with superior evaders , 2009, GEC '09.

[31]  Howard M. Schwartz,et al.  Multi-Agent Machine Learning: A Reinforcement Approach , 2014 .

[32]  Genshe Chen,et al.  Multi-Pursuer Multi-Evader Pursuit-Evasion Games with Jamming Confrontation , 2007, J. Aerosp. Comput. Inf. Commun..

[33]  Pierre T. Kabamba,et al.  Pursuit-evasion games in the presence of a line segment obstacle , 2014, 53rd IEEE Conference on Decision and Control.

[34]  M.A. Wiering,et al.  Reinforcement Learning in Continuous Action Spaces , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[35]  Naomi Ehrich Leonard,et al.  Dynamics of pursuit and evasion in a heterogeneous herd , 2014, 53rd IEEE Conference on Decision and Control.

[36]  Jie Dong,et al.  Strategies of Pursuit-Evasion Game Based on Improved Potential Field and Differential Game Theory for Mobile Robots , 2012, 2012 Second International Conference on Instrumentation, Measurement, Computer, Communication and Control.

[37]  Li-Xin Wang,et al.  A Course In Fuzzy Systems and Control , 1996 .

[38]  Jingtai Liu,et al.  Research on Pursuit-evasion games with multiple heterogeneous pursuers and a high speed evader , 2015, The 27th Chinese Control and Decision Conference (2015 CCDC).

[39]  Dongxu Li,et al.  A Hierarchical Approach To Multi-Player Pursuit-Evasion Differential Games , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[40]  Dongxu Li,et al.  Better cooperative control with limited look-ahead , 2006, 2006 American Control Conference.

[41]  Kenji Doya,et al.  Reinforcement Learning in Continuous Time and Space , 2000, Neural Computation.

[42]  Richard B. Vinter,et al.  A decomposition technique for pursuit evasion games with many pursuers , 2013, 52nd IEEE Conference on Decision and Control.

[43]  Panagiotis Tsiotras,et al.  An asymmetric version of the two car pursuit-evasion game , 2014, 53rd IEEE Conference on Decision and Control.

[44]  Howard M. Schwartz,et al.  Decentralized learning in multiple pursuer-evader Markov games , 2011, 2011 19th Mediterranean Conference on Control & Automation (MED).

[45]  Jason M. O'Kane,et al.  A complete algorithm for visibility-based pursuit-evasion with multiple pursuers , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[46]  Genshe Chen,et al.  A Decentralized Approach to Pursuer-Evader Games with Multiple Superior Evaders in Noisy Environments , 2007, 2007 IEEE Aerospace Conference.