Levels of Realism for Cooperative Multi-Agent Reinforcement Learning

Training agents in a virtual crowd to achieve a task can be accomplished by allowing the agents to learn by trial-and-error and by sharing information with other agents. Since sharing enables agents to potentially reach optimal behavior more quickly, what type of sharing is best to use to achieve the quickest learning times? This paper categorizes sharing into three categories: realistic, unrealistic, and no sharing. Realistic sharing is defined as sharing that takes place amongst agents within close proximity and unrealistic sharing allows agents to share regardless of physical location. This paper demonstrates that all sharing methods converge to similar policies and that the differences between the methods are determined by analyzing the learning rates, communication frequencies, and total run times. Results show that the unrealistic-centralized sharing method --- where agents update a common learning module --- is the most effective of the sharing methods tested.

[1]  Lisa Torrey,et al.  Crowd Simulation Via Multi-Agent Reinforcement Learning , 2010, AIIDE.

[2]  Minoru Asada,et al.  Behavior acquisition by multi-layered reinforcement learning , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[3]  Xiong Li,et al.  Multi-Agent Cooperative Reinforcement Learning in 3D Virtual World , 2010, ICSI.

[4]  ndez,et al.  Multi-agent reinforcement learning for simulating pedestrian navigation , 2011, ALA-11 2011.

[5]  Ming Tan,et al.  Multi-Agent Reinforcement Learning: Independent versus Cooperative Agents , 1997, ICML.

[6]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[8]  Fernando Fernández,et al.  Multi-agent Reinforcement Learning for Simulating Pedestrian Navigation , 2011, ALA.

[9]  Fabrício Enembreck,et al.  Interaction Models for Multiagent Reinforcement Learning , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[10]  Matthew E. Taylor,et al.  Adaptive and Learning Agents, Second Workshop, ALA 2009, Held as Part of the AAMAS 2009 Conference in Budapest, Hungary, May 12, 2009, Revised Selected Papers , 2010, ALA.

[11]  Minoru Asada,et al.  Multi-controller fusion in multi-layered reinforcement learning , 2001, Conference Documentation International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI 2001 (Cat. No.01TH8590).

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.