Which Types of Learning Make a Simple Game Complex?

The present study focuses on a class of games with reinforcement-learning agents that adaptively choose their actions to locally maximize their rewards. By analyzing a limit model with a special type of learning, previous studies suggested that dynamics of games with learners may become chaotic. We evaluated the generality of this model by analyzing the consistency of this limit model in comparison with two other approaches, agent-based simulation and the Markov process model. Our analysis showed inconsistency between the limit model and two other models with more general reinforcement learning. This suggests that reinforcement learning does not lead to complex dynamics in games with learners.

[1]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  E. Seneta.  Non-negative Matrices and Markov Chains , 2008 .

[4]  Karl Pearson F.R.S..  X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling , 2009 .

[5]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 2019, Wiley Series in Probability and Statistics.

[6]  M. Nowak.  Stochastic strategies in the Prisoner's Dilemma , 1990 .

[7]  Eizo Akiyama,et al.  Chaos in learning a simple two-person game , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[8]  James P. Crutchfield,et al.  Stability and diversity in collective adaptation , 2004, nlin/0408039.

[9]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[10]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[11]  Stephen Wolfram,et al.  A New Kind of Science , 2003, Artificial Life.

[12]  Colin Camerer.  Behavioural studies of strategic thinking in games , 2003, Trends in Cognitive Sciences.

[13]  Tobias Galla,et al.  Intrinsic noise in game dynamical learning. , 2009, Physical review letters.

[14]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[15]  A. Roth,et al.  Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term* , 1995 .

[16]  Robert H. Crites,et al.  Multiagent reinforcement learning in the Iterated Prisoner's Dilemma. , 1996, Bio Systems.

[17]  J. Crutchfield,et al.  Coupled replicator equations for the dynamics of learning in multiagent systems. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.