Towards collaborative and adversarial learning: a case study in robotic soccer

Soccer is a rich domain for the study of multiagent learning issues. Not only must the players learn lower-level skills, but they must also learn to work together and to adapt to the behaviors of different opponents. We are using a robotic soccer system to study these different types of multiagent learning: low-level skills, collaborative and adversarial. Here we describe in detail our experimental framework. We present a learned, robust, low-level behavior that is necessitated by the multiagent nature of the domain, viz. shooting a moving ball. We then discuss the issues that arise as we extend the learning scenario to require collaborative and adversarial learning.

[1]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[2]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[3]  L. Shepp,et al.  OPTIMAL PATHS FOR A CAR THAT GOES BOTH FORWARDS AND BACKWARDS , 1990 .

[4]  Jean-Claude Latombe.  A Fast Path Planner for a Car-Like Indoor Mobile Robot , 1991, AAAI.

[5]  Michael K. Sahota,et al.  Real-time intelligent behaviour in dynamic environments : soccer-playing robots , 1993 .

[6]  Minoru Asada,et al.  Coordination of multiple behaviors acquired by a vision-based reinforcement learning , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[7]  I. Noda.  Soccer Server: a simulator of RoboCup , 1995 .

[8]  Manuela M. Veloso,et al.  Beating a Defender in Robotic Soccer: Memory-Based Learning of a Continuous Function , 1995, NIPS.

[9]  Alan K. Mackworth,et al.  Real-time control of soccer-playing robots using off-board vision: the dynamite testbed , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[10]  Gerhard Weiß,et al.  Distributed reinforcement learning , 1995, Robotics Auton. Syst..

[11]  Peter Stone.  Broad Learning from Narrow Training: A Case Study in Robotic Soccer , 1995 .

[12]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[13]  Sorin Achim and Peter Stone and Manuela Veloso.  Building a Dedicated Robotic Soccer System , 1996 .

[14]  Hiroaki Kitano,et al.  RoboCup: The Robot World Cup Initiative , 1997, AGENTS '97.

[15]  S. F. Actory,et al.  Personal correspondence , 1997 .

[16]  Hiroaki Kitano,et al.  RoboCup: A Challenge Problem for AI , 1997, AI Mag..

[17]  Manuela M. Veloso,et al.  Layered Approach to Learning Client Behaviors in the Robocup Soccer Server , 1998, Appl. Artif. Intell..

[18]  Katia P. Sycara,et al.  Bayesian learning in negotiation , 1998, Int. J. Hum. Comput. Stud..

[19]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.