Simultaneous learning to acquire competitive behaviors in multi-agent system based on a modular learning system

Existing reinforcement learning approaches have been suffering from the policy alternation of others in multiagent dynamic environments. A typical example is the case of RoboCup competitions because other agent behaviors may cause sudden changes in state transition probabilities in which constancy is needed for the learning to converge. The keys for simultaneous learning to acquire competitive behaviors in such an environment are: a modular learning system for adaptation to the policy alternation of others; and an introduction of macro actions for simultaneous learning to reduce the search space. This paper presents a method of modular learning in a multiagent environment in which the learning agents can simultaneously learn their behaviors and adapt themselves to the situations as a consequence of the others' behaviors.

[1]  Satinder P. Singh,et al.  Reinforcement Learning with a Hierarchy of Abstract Models , 1992, AAAI.

[2]  Sridhar Mahadevan,et al.  Robot Learning , 1993 .

[3]  Stefano Nolfi,et al.  Self-Organization of Modules and Their Hierarchy in Robot Learning Problems: A Dynamical Systems App , 1997 .

[4]  Minoru Asada,et al.  Cooperative behavior acquisition by asynchronous policy renewal that enables simultaneous learning in multiagent environment , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[6]  Minoru Asada,et al.  Multi-module learning system for behavior acquisition in multi-agent environment , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[8]  Peter Stone,et al.  Scaling Reinforcement Learning toward RoboCup Soccer , 2001, ICML.

[9]  Henrik I. Christensen,et al.  Multi-agent reinforcement learning: using macro actions to learn a mating task , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[10]  Minoru Asada,et al.  Cooperative Behavior Acquisition for Mobile Robots in Dynamically Changing Real Worlds Via Vision-Based Reinforcement Learning and Development , 1999, Artif. Intell..

[11]  Minoru Asada,et al.  Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning , 2005, Machine Learning.

[12]  Peter Stone,et al.  Progress in Learning 3 vs. 2 Keepaway , 2003, RoboCup.

[13]  Satinder P. Singh,et al.  The Efficient Learning of Multiple Task Sequences , 1991, NIPS.

[14]  Satinder Singh Transfer of learning by composing solutions of elemental sequential tasks , 2004, Machine Learning.