Sequence-Form and Evolutionary Dynamics: Realization Equivalence to Agent Form and Logit Dynamics

Evolutionary game theory provides the principal tools to model the dynamics of multi–agent learning algorithms. While there is a long–standing literature on evolutionary game theory in strategic–form games, in the case of extensive–form games few results are known and the exponential size of the representations currently adopted makes the evolutionary analysis of such games unaffordable. In this paper, we focus on dynamics for the sequence form of extensive–form games, providing three dynamics: one realization equivalent to the normal–form logit dynamic, one realization equivalent to the agent–form replicator dynamic, and one realization equivalent to the agent–form logit dynamic. All the considered dynamics require polynomial time and space, providing an exponential compression w.r.t. the dynamics currently known and providing thus tools that can be effectively employed in practice. Moreover, we use our tools to compare the agent–form and normal– form dynamics and to provide new "hybrid" dynamics.

[1]  H W Kuhn Extensive Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Marcello Restelli,et al.  Efficient Evolutionary Dynamics with Extensive-Form Games , 2013, AAAI.

[3]  Karl Tuyls,et al.  An Evolutionary Dynamical Analysis of Multi-Agent Learning in Iterated Games , 2005, Autonomous Agents and Multi-Agent Systems.

[4]  L. Blume The Statistical Mechanics of Strategic Interaction , 1993 .

[5]  Karl Tuyls,et al.  Theoretical Advantages of Lenient Learners: An Evolutionary Game Theoretic Perspective , 2008, J. Mach. Learn. Res..

[6]  B. Stengel,et al.  Efficient Computation of Behavior Strategies , 1996 .

[7]  Diodato Ferraioli,et al.  Logit dynamics: a model for bounded rationality , 2013, SECO.

[8]  Rong Yang,et al.  Computing optimal strategy against quantal response in security games , 2012, AAMAS.

[9]  Marc Lanctot,et al.  Further developments of extensive-form replicator dynamics using the sequence-form representation , 2014, AAMAS.

[10]  Ross Cressman,et al.  Evolutionary and Dynamic stability in Symmetric Evolutionary Games with Two Independent Decisions , 2000, IGTR.

[11]  Karl Tuyls,et al.  Evolutionary Dynamics of Multi-Agent Learning: A Survey , 2015, J. Artif. Intell. Res..

[12]  William H. Sandholm,et al.  Population Games And Evolutionary Dynamics , 2010, Economic learning and social evolution.

[13]  R. Cressman Evolutionary Dynamics and Extensive Form Games , 2003 .

[14]  R. Selten Reexamination of the perfectness concept for equilibrium points in extensive games , 1975, Classics in Game Theory.

[15]  Marcello Restelli,et al.  Evolutionary Dynamics of Q-Learning over the Sequence Form , 2014, AAAI.

[16]  R. McKelvey,et al.  Quantal Response Equilibria for Extensive Form Games , 1998 .

[17]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[18]  Simon Parsons,et al.  What evolutionary game theory tells us about multiagent learning , 2007, Artif. Intell..

[19]  Dries Vermeulen,et al.  The reduced form of a game , 1998, Eur. J. Oper. Res..

[20]  Ross Cressman,et al.  Subgame Monotonicity in Extensive Form Evolutionary Games , 2000, Games Econ. Behav..