A hybrid genetic programming decision making system for RoboCup soccer simulation

In this contribution we propose a hybrid genetic programming approach for evolving a decision making system in the domain of RoboCup Soccer (Simulation League). Genetic programming has been rarely used in this domain in the past, due to the difficulties and restrictions of the soccer simulation. The real-time requirements of robot soccer and the lengthy evaluation time even for simulated games provide a formidable obstacle to the application of evolutionary approaches. Our new method uses two evolutionary phases, each of which compensating for restrictions and limitations of the other. The first phase produces some evolved GP individuals applying an off-game evaluation system which can be trained on snapshots of game situations as they actually happened in earlier games, and corresponding decisions tagged as correct or wrong. The second phase uses the best individuals of the first phase as input to run another GP system to evolve players in a real game environment where the quality of decisions is evaluated through winning or losing during real-time runs of the simulator. We benchmark the new system against a baseline system used by most simulation league teams, as well as against winning systems of the 2016 tournament.

[1]  Astro Teller,et al.  Evolving Team Darwin United , 1998, RoboCup.

[2]  Hitoshi Iba.  Emergent Cooperation for Multiple Agents Using Genetic Programming , 1996, PPSN.

[3]  Sean Luke,et al.  Cooperative Multi-Agent Learning: The State of the Art , 2005, Autonomous Agents and Multi-Agent Systems.

[4]  Jonatan Aronsson,et al.  Genetic Programming of Multi-agent System in the RoboCup Domain , 2003 .

[5]  I. Noda.  Soccer Server: a simulator of RoboCup , 1995 .

[6]  Sean Luke,et al.  Genetic Programming Produced Competitive Soccer Softbot Teams for RoboCup97 , 1998 .

[7]  H. Levent Akin,et al.  Solving Multi-agent Decision Problems Modeled as Dec-POMDP: A Robot Soccer Case Study , 2012, RoboCup.

[8]  Adil Qureshi,et al.  Evolving agents , 1996 .

[9]  David Andre.  The Automatic Programming of Agents that Learn Mental Models and Create Simple Plans of Action , 1995, IJCAI.

[10]  Jacques Ferber,et al.  Multi-agent systems - an introduction to distributed artificial intelligence , 1999 .

[11]  Sandip Sen,et al.  Crossover Operators for Evolving A Team , 1997 .

[12]  James A. Hendler,et al.  Co-evolving Soccer Softbot Team Coordination with Genetic Programming , 1997, RoboCup.

[13]  Lee Spector,et al.  Evolving teamwork and coordination with genetic programming , 1996 .

[14]  Kenneth Sundaraj,et al.  A survey on team strategies in robot soccer: team strategies and role description , 2013, Artificial Intelligence Review.

[15]  Wolfgang Banzhaf,et al.  Evolutionary Computation and Genetic Programming , 2013 .

[16]  Michael Wooldridge,et al.  Multi-Agent systems: an introduction , 2001 .

[17]  Amir Tavafi,et al.  MarliK 2012 Soccer 2D Simulation Team Description Paper , 2012 .

[18]  Peter Nordin,et al.  An On-Line Method to Evolve Behavior and to Control a Miniature Robot in Real Time with Genetic Programming , 1996, Adapt. Behav..

[19]  Dario Floreano,et al.  Evolving Team Compositions by Agent Swapping , 2013, IEEE Transactions on Evolutionary Computation.