Stability and Long Run Equilibrium in Stochastic Fictitious Play

In this paper we develop methods to analyze the long run behavior of models with multiple stable equilibria, and we apply them to a well known model of learning in games. Our methods apply to discrete-time continuous-state stochastic models, and as a particular application in we study a model of stochastic flctitious play. We focus on a variant of this model in which agents’ payofis are subject to random shocks and they discount past observations exponentially. We analyze the behavior of agents’ beliefs as the discount rate on past information becomes small but the payofi shock variance remains flxed. We show that agents tend to be drawn toward an equilibrium, but occasionally the stochastic shocks lead agents to endogenously shift between equilibria. We then calculate the invariant distribution of players’ beliefs, and use it to determine the most likely outcome observed in long run. Our application shows that by making some slight changes to a standard learning model, we can derive an equilibrium selection criterion similar to stochastic evolutionary models but with some important difierences.

[1]  M. Hirsch,et al.  Stochastic approximation algorithms with constant step size whose average is cooperative , 1999 .

[2]  Drew Fudenberg,et al.  Learning Purified Mixed Equilibria , 2000, J. Econ. Theory.

[3]  R. Sarin,et al.  Payoff Assessments without Probabilities: A Simple Dynamic Model of Choice , 1999 .

[4]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[5]  Daniel Friedman,et al.  Individual Learning in Normal Form Games: Some Laboratory Results☆☆☆ , 1997 .

[6]  P. Dupuis,et al.  Stochastic approximation and large deviations: upper bounds and w.p.1 convergence , 1989 .

[7]  Josef Hofbauer,et al.  Evolution and Learning in Games with Randomly Disturbed Payoffs , 2001 .

[8]  David P. Myatt,et al.  Adaptive play by idiosyncratic agents , 2004, Games Econ. Behav..

[9]  Richard T. Boylan.  Continuous Approximation of Dynamical Systems with Randomly Matched Individuals , 1995 .

[10]  R. Rob,et al.  Learning, Mutation, and Long Run Equilibria in Games , 1993 .

[11]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[12]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[13]  P. Dupuis,et al.  Stochastic Approximations via Large Deviations: Asymptotic Properties , 1985 .

[14]  Youngse Kim,et al.  Equilibrium Selection inn-Person Coordination Games , 1996 .

[15]  H. Peyton Young,et al.  Stochastic Evolutionary Game Dynamics , 1990 .

[16]  R. McKelvey,et al.  Quantal Response Equilibria for Normal Form Games , 1995 .

[17]  M. Benaïm,et al.  Deterministic Approximation of Stochastic Evolution in Games , 2003 .

[18]  M. Hirsch,et al.  Mixed Equilibria and Dynamical Systems Arising from Fictitious Play in Perturbed Games , 1999 .

[19]  L. Shapley.  SOME TOPICS IN TWO-PERSON GAMES , 1963 .

[20]  H. Young,et al.  The Evolution of Conventions , 1993 .

[21]  David M. Kreps,et al.  Learning Mixed Equilibria , 1993 .

[22]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[23]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[24]  D. Fudenberg,et al.  Evolutionary Dynamics with Aggregate Shocks , 1992 .

[25]  D. Fudenberg,et al.  Consistency and Cautious Fictitious Play , 1995 .

[26]  H. Young,et al.  Learning dynamics in games with stochastic perturbations , 1995 .

[27]  L. Samuelson,et al.  Musical Chairs: Modeling Noisy Evolution , 1995 .

[28]  H. Kushner.  Robustness and Approximation of Escape Times and Large Deviations Estimates for Systems with Small Noise Effects , 1984 .

[29]  Michel Bena.  Convergence with probability one of stochastic approximation algorithms whose average is cooperative , 2000 .