A survey on multi-agent reinforcement learning: Coordination problems

Learning in multiagent system needs to solve the complexity of the task, so multiagent reinforcement learning has been focused on theoretical research and various applications. In multiagent reinforcement learning, agents can be compete or cooperate to accomplish the goal. For cooperative multiagent reinforcement learning(CMRL), agents have to coordinate with other agents. Therefore, coordination problems in CMRL are getting more and more important because of increasing the number of agents and actions. There are several algorithms dealt with cooperative multiagent reinforcement learning using stochastic games, coordinated graph, and so on. These algorithms have some assumptions to coordinate each other, however assumptions are not consistent with characteristics of the multiagent system. In this paper, we provide a survey on coordination problems in cooperative multiagent reinforcement learning, and propose new approach to solve coordination problems.

[1]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[2]  Richard M. Murray,et al.  Information flow and cooperative control of vehicle formations , 2004, IEEE Transactions on Automatic Control.

[3]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[4]  Keith B. Hall,et al.  Correlated Q-Learning , 2003, ICML.

[5]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[6]  Michael L. Littman,et al.  Value-function reinforcement learning in Markov games , 2001, Cognitive Systems Research.

[7]  Daniel Kudenko,et al.  Reinforcement learning of coordination in heterogeneous cooperative multi-agent systems , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[8]  M. Dufwenberg.  Game theory. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[9]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[10]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[12]  Daniel Kudenko,et al.  Reinforcement Learning Approaches to Coordination in Cooperative Multi-agent Systems , 2002, Adaptive Agents and Multi-Agents Systems.

[13]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[14]  Nikos A. Vlassis,et al.  Sparse cooperative Q-learning , 2004, ICML.

[15]  Martin Lauer,et al.  An Algorithm for Distributed Reinforcement Learning in Cooperative Multi-Agent Systems , 2000, ICML.

[16]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[17]  Michail G. Lagoudakis,et al.  Coordinated Reinforcement Learning , 2002, ICML.

[18]  Ming Tan,et al.  Multi-Agent Reinforcement Learning: Independent versus Cooperative Agents , 1997, ICML.

[19]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[20]  Gerhard Weiss,et al.  Multiagent Systems , 1999 .

[21]  Ella M. Atkins,et al.  Distributed multi‐vehicle coordinated control via local information exchange , 2007 .

[22]  Agostino Poggi,et al.  Multiagent Systems , 2006, Intelligenza Artificiale.

[23]  Csaba Szepesvári,et al.  A Generalized Reinforcement-Learning Model: Convergence and Applications , 1996, ICML.

[24]  A. Rubinstein,et al.  A Course in Game Theory , 1995 .

[25]  Bart De Schutter,et al.  A Comprehensive Survey of Multiagent Reinforcement Learning , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[26]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .