Online Cooperative Behavior Planning Using a Tree Search Method in the RoboCup Soccer Simulation

In this paper, we propose a tree search approach to generate and evaluate cooprative behavior online in multiagent systems. It was difficult to apply a tree search methodology to tasks that the state-action space is continuous and requires realtimeness. However, it has become possible to apply such an approach since the computational resources became more powerful today. We applied a tree search method to the Robo Cup soccer 2D simulation and analyzed its effect by evaluating the team performance.

[1]  Jiang Chen,et al.  An application in RoboCup combining Q-learning with adversarial planning , 2002, Proceedings of the 4th World Congress on Intelligent Control and Automation (Cat. No.02EX527).

[2]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[3]  Julian Togelius,et al.  The 2009 Mario AI Competition , 2010, IEEE Congress on Evolutionary Computation.

[4]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[5]  Hidehisa Akiyama,et al.  Multi-agent Positioning Mechanism in the Dynamic Environment , 2008, RoboCup.

[6]  Hitoshi Matsubara,et al.  Soccer Server and Researches on Multi-Agent Systems , 1996 .

[7]  Martin A. Riedmiller,et al.  A Case Study on Improving Defense Behavior in Soccer Simulation 2D: The NeuroHassle Approach , 2009, RoboCup.

[8]  Luís Paulo Reis,et al.  COACH UNILANG - A Standard Language for Coaching a (Robo)Soccer Team , 2001, RoboCup.

[9]  Luís Paulo Reis,et al.  Setplays: achieving coordination by the appropriate use of arbitrary pre-defined flexible plans and inter-robot communication , 2007, ROBOCOMM.

[10]  Julian Togelius,et al.  The Mario AI Benchmark and Competitions , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[11]  Manuela M. Veloso,et al.  Planning for Distributed Execution through Use of Probabilistic Opponent Models , 2002, AIPS.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Katsumi Nitta,et al.  Training of Agent Positioning Using Human's Instruction , 2007, J. Adv. Comput. Intell. Intell. Informatics.

[14]  Manuela M. Veloso,et al.  Task Decomposition, Dynamic Role Assignment, and Low-Bandwidth Communication for Real-Time Strategic Teamwork , 1999, Artif. Intell..

[15]  Peter Stone,et al.  Reinforcement Learning for RoboCup Soccer Keepaway , 2005, Adapt. Behav..

[16]  Luís Paulo Reis,et al.  Situation Based Strategic Positioning for Coordinating a Team of Homogeneous Agents , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.