A numerical analysis of the evolutionary stability of learning rules

In this paper I define an evolutionary stability criterion for learning rules. Using Monte Carlo simulations, I then apply this criterion to a class of learning rules that can be represented by Camerer and Ho's (1999) model of learning. This class contains perturbed versions of reinforcement and belief learning as special cases. A large population of individuals with learning rules in this class are repeatedly rematched for a finite number of periods and play one out of four symmetric two-player games. Belief learning is the only learning rule which is evolutionarily stable in almost all cases, whereas reinforcement learning is unstable in almost all cases. I also find that in certain games, the stability of intermediate learning rules hinges critically on a parameter of the model and the relative payoffs.

[1]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[2]  Dana Heller,et al.  An evolutionary approach to learning in a changing environment , 2004, J. Econ. Theory.

[3]  Günther Palm,et al.  Evolutionary stable strategies and game dynamics for n-person games , 1984 .

[4]  Nathaniel T. Wilcox,et al.  Theories of Learning in Games and Heterogeneity Bias , 2006 .

[5]  Dale O. Stahl,et al.  Action-Reinforcement Learning Versus Rule Learning , 2000 .

[6]  Leigh Tesfatsion,et al.  Agent-Based Computational Economics: Growing Economies From the Bottom Up , 2002, Artificial Life.

[7]  C. Hommes.  Heterogeneous Agent Models in Economics and Finance , 2005 .

[8]  Colin Camerer,et al.  Experience‐weighted Attraction Learning in Normal Form Games , 1999 .

[9]  E. Hopkins.  Learning, Matching and Aggregation , 1995 .

[10]  J. M. Smith.  The theory of games and the evolution of animal conflicts. , 1974, Journal of theoretical biology.

[11]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[12]  Alan W. Beggs,et al.  On the convergence of reinforcement learning , 2005, J. Econ. Theory.

[13]  Dearborn.  Animal Intelligence: An Experimental Study of the Associative Processes in Animals , 1900 .

[14]  The Evolution of Algorithmic Learning Rules: A Global Stability Result , 1996 .

[15]  R. R. Bush,et al.  A Mathematical Model for Simple Learning , 1951 .

[16]  Ulrich Berger,et al.  Fictitious play in 2×n games , 2005, J. Econ. Theory.

[17]  Arthur J. Robson,et al.  The Biological Basis of Economic Behavior , 2001 .

[18]  Colin Camerer.  Behavioral Game Theory: Experiments in Strategic Interaction , 2003 .

[19]  C. Harley.  Learning the evolutionarily stable strategy. , 1981, Journal of theoretical biology.

[20]  Teck-Hua Ho,et al.  Sophisticated Experience-Weighted Attraction Learning and Strategic Teaching in Repeated Games , 2002, J. Econ. Theory.

[21]  Colin F. Camerer,et al.  Functional EWA: A One-parameter Theory of Learning in Games , 2002 .

[22]  I. Erev,et al.  LEARNING STRATEGIES , 2010 .

[23]  William H. Sandholm,et al.  ON THE GLOBAL CONVERGENCE OF STOCHASTIC FICTITIOUS PLAY , 2002 .

[24]  Martin Posch,et al.  Attainability of boundary points under reinforcement learning , 2005, Games Econ. Behav..

[25]  E. Thorndike.  “Animal Intelligence” , 1898, Nature.

[26]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[27]  E. Hopkins.  Two Competing Models of How People Learn in Games (first version) , 1999 .

[28]  Timothy C. Salmon.  An Evaluation of Econometric Models of Adaptive Learning , 2001 .

[29]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .