Complex dynamics in learning complicated games

Game theory is the standard tool used to model strategic interactions in evolutionary biology and social science. Traditionally, game theory studies the equilibria of simple games. However, is this useful if the game is complicated, and if not, what is? We define a complicated game as one with many possible moves, and therefore many possible payoffs conditional on those moves. We investigate two-person games in which the players learn based on a type of reinforcement learning called experience-weighted attraction (EWA). By generating games at random, we characterize the learning dynamics under EWA and show that there are three clearly separated regimes: (i) convergence to a unique fixed point, (ii) a huge multiplicity of stable fixed points, and (iii) chaotic behavior. In case (iii), the dimension of the chaotic attractors can be very high, implying that the learning dynamics are effectively random. In the chaotic regime, the total payoffs fluctuate intermittently, showing bursts of rapid change punctuated by periods of quiescence, with heavy tails similar to what is observed in fluid turbulence and financial markets. Our results suggest that, at least for some learning algorithms, there is a large parameter regime for which complicated strategic interactions generate inherently unpredictable behavior that is best described in the language of dynamical systems theory.

[1]  Andrew McLennan,et al.  Asymptotic expected number of Nash equilibria of two-player normal form games , 2005, Games Econ. Behav..

[2]  Rama Cont,et al.  Comment on "Turbulent cascades in foreign exchange markets" , 1996, cond-mat/9607120.

[3]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[4]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[5]  J. Crutchfield,et al.  Coupled replicator equations for the dynamics of learning in multiagent systems. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[6]  Colin Camerer,et al.  Experience‐weighted Attraction Learning in Normal Form Games , 1999 .

[7]  Tobias Galla,et al.  Random replicators with asymmetric couplings , 2005, cond-mat/0508174.

[8]  W. Brock,et al.  Heterogeneous beliefs and routes to chaos in a simple asset pricing model , 1998 .

[9]  E. Lorenz.  Atmospheric Predictability as Revealed by Naturally Occurring Analogues , 1969 .

[10]  Tobias Galla,et al.  Two-population replicator dynamics and number of Nash equilibria in matrix games , 2006, q-bio/0608032.

[11]  Tobias Galla,et al.  Fixation and escape times in stochastic game learning , 2011, ArXiv.

[12]  Eizo Akiyama,et al.  Chaos in learning a simple two-person game , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[13]  H. Young,et al.  Individual Strategy and Social Structure: An Evolutionary Theory of Institutions , 1999 .

[14]  Daniele Vilone,et al.  Chaos and unpredictability in evolutionary dynamics in discrete time. , 2011, Physical review letters.

[15]  Martin Weigt,et al.  Entropy and typical properties of Nash equilibria in two-player games , 1999 .

[16]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[17]  J. Neumann,et al.  The Theory of Games and Economic Behaviour , 1944 .

[18]  J. Peinke,et al.  Turbulent cascades in foreign exchange markets , 1996, Nature.

[19]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[20]  BRIAN SKYRMS,et al.  Chaos in game dynamics , 1992, J. Log. Lang. Inf..

[21]  ROBERT M. MAY,et al.  Will a Large Complex System be Stable? , 1972, Nature.

[22]  C. Dominicis.  Dynamics as a substitute for replicas in systems with quenched random impurities , 1978 .

[23]  A.C.C. Coolen,et al.  Chapter 15 Statistical mechanics of recurrent neural networks II — Dynamics , 2000, cond-mat/0006011.

[24]  Ton Coolen,et al.  The mathematical theory of minority games , 2005 .

[25]  Opper,et al.  Phase transition and 1/f noise in a game dynamical model. , 1992, Physical review letters.

[26]  ปิยดา สมบัติวัฒนา.  Behavioral Game Theory: Experiments in Strategic Interaction , 2013 .

[27]  Tobias Galla,et al.  Intrinsic noise in game dynamical learning. , 2009, Physical review letters.

[28]  H. Peyton Young,et al.  Individual Strategy and Social Structure , 2020 .

[29]  H P Young,et al.  On the impossibility of predicting the behavior of rational agents , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[30]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[31]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[32]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[33]  J. Hofbauer,et al.  Uncoupled Dynamics Do Not Lead to Nash Equilibrium , 2003 .

[34]  D. Sherrington,et al.  Dynamics of a spherical minority game , 2003, cond-mat/0303615.

[35]  Opper,et al.  Replicators with random interactions: A solvable model. , 1989, Physical review. A, General physics.

[36]  R. Axelrod,et al.  Evolutionary Dynamics , 2004 .

[37]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[38]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[39]  Teck-Hua Ho,et al.  Self-tuning experience weighted attraction learning in games , 2007, J. Econ. Theory.