An Evolutionary Game Theoretic Perspective on Learning in Multi-Agent Systems

In this paper we revise Reinforcement Learning and adaptiveness in Multi-Agent Systems from an Evolutionary Game Theoretic perspective. More precisely we show there is a triangular relation between the fields of Multi-Agent Systems, Reinforcement Learning and Evolutionary Game Theory. We illustrate how these new insights can contribute to a better understanding of learning in MAS and to new improved learning algorithms. All three fields are introduced in a self-contained manner. Each relation is discussed in detail with the necessary background information to understand it, along with major references to relevant work.

[1]  Herbert Gintis,et al.  Game Theory Evolving: A Problem-Centered Introduction to Modeling Strategic Interaction - Second Edition , 2009 .

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Peter Stone,et al.  Layered Learning in Multiagent Systems , 1997, AAAI/IAAI.

[4]  Karl Tuyls,et al.  On a Dynamical Analysis of Reinforcement Learning in Games: Emergence of Occam's Razor , 2003, CEEMAS.

[5]  John Loch,et al.  Using Eligibility Traces to Find the Best Memoryless Policy in Partially Observable Markov Decision Processes , 1998, ICML.

[6]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[7]  Barbara Messing,et al.  An Introduction to MultiAgent Systems , 2002, Künstliche Intell..

[8]  Gunes Ercal,et al.  On No-Regret Learning, Fictitious Play, and Nash Equilibrium , 2001, ICML.

[9]  Tom Lenaerts,et al.  A selection-mutation model for q-learning in multi-agent systems , 2003, AAMAS '03.

[10]  Dietrich Braess,et al.  Über ein Paradoxon aus der Verkehrsplanung , 1968, Unternehmensforschung.

[11]  Fernando Redondo.  Game Theory and Economics , 2001 .

[12]  Ann Nowé,et al.  Social Agents Playing a Periodical Policy , 2001, ECML.

[13]  M. Hirsch,et al.  Differential Equations, Dynamical Systems, and Linear Algebra , 1974 .

[14]  D. Stauffer.  Life, Love and Death: Models of Biological Reproduction and Aging , 1999 .

[15]  Gerhard Weiss,et al.  Multiagent Systems , 1999 .

[16]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[17]  Mark D. Pendrith,et al.  An Analysis of Direct Reinforcement Learning in Non-Markovian Domains , 1998, ICML.

[18]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[19]  John N. Tsitsiklis,et al.  Asynchronous stochastic approximation and Q-learning , 1994, Mach. Learn..

[20]  T. D. Schneider,et al.  Evolution of biological information. , 2000, Nucleic acids research.

[21]  Theodore J. Perkins,et al.  On the Existence of Fixed Points for Q-Learning and Sarsa in Partially Observable Domains , 2002, ICML.

[22]  Leonid Sheremetov,et al.  Weiss, Gerhard. Multiagent Systems a Modern Approach to Distributed Artificial Intelligence , 2009 .

[23]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning in Stochastic Games , 1999, ICML 1999.

[24]  J. Weibull.  What have we learned from Evolutionary Game Theory so far , 1998 .

[25]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[26]  Tom Lenaerts,et al.  Towards a relation between learning agents and evolutionary dynamics , 2002 .

[27]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[28]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[29]  Peter Stone,et al.  Layered learning in multiagent systems - a winning approach to robotic soccer , 2000, Intelligent robotics and autonomous agents.

[30]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[31]  Michael Luck,et al.  Agent technology: Enabling next generation computing , 2003 .

[32]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[33]  Bernard Manderick,et al.  Extended Replicator Dynamics as a Key to Reinforcement Learning in Multi-agent Systems , 2003, ECML.

[34]  L. Samuelson.  Evolutionary Games and Equilibrium Selection , 1997 .

[35]  Gerhard Weiss,et al.  Multiagent systems: a modern approach to distributed artificial intelligence , 1999 .

[36]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[37]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[38]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[39]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[40]  D. E. Matthews.  Evolution and the Theory of Games , 1977 .

[41]  Gerhard Weiß,et al.  Distributed reinforcement learning , 1995, Robotics Auton. Syst..

[42]  D. Serra,et al.  Game theory and economics , 2003 .