Reinforcement learning in the El Farol model

Abstract The paper applies the approach of reinforcement learning to Arthur’s [American Economic Review 84 (1994a) 406] El Farol problem, where repeatedly a population of agents decides to go to a bar or stay home, and going is enjoyable if, and only if, the bar is not crowded. The numerical simulations show that the learning process is fairly efficient in that attendances tend to fluctuate around the bar’s capacity. On the other hand, the shape of the long-run equilibrium frequency distribution is usually quite distinct from the Nash equilibrium solutions and furthermore rather sensitive to some of the model’s specification details.

[1]  Klaus Krickeberg,et al.  Markov learning models for multiperson interactions , 1962 .

[2]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .

[3]  R. R. Bush,et al.  A Mathematical Model for Simple Learning , 1951 .

[4]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[5]  Daniel Friedman,et al.  A comparison of learning and replicator dynamics using experimental data , 1998 .

[6]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[7]  Tilman Börgers,et al.  Naive Reinforcement Learning With Endogenous Aspirations , 2000 .

[8]  Mark Walker,et al.  An Experiment to Evaluate Bayesian Learning of Nash Equilibrium Play , 2001, Games Econ. Behav..

[9]  W. Arthur.  Inductive Reasoning and Bounded Rationality , 1994 .

[10]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[11]  Bernard Walliser,et al.  A spectrum of equilibration processes in game theory , 1998 .

[12]  T. Dieckmann.  Stochastic Learning and the Evolution of Conventions , 1998 .

[13]  A. Roth,et al.  Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term* , 1995 .

[14]  W. Brian Arthur,et al.  On designing economic agents that behave like human agents , 1993 .

[15]  Games, Economic Dynamics, and Time Series Analysis , 1982 .

[16]  T. Brenner.  Modelling Learning in Economics , 1999 .

[17]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[18]  Bruce Edmonds,et al.  Modeling Socially Intelligent Agents , 1998, Appl. Artif. Intell..

[19]  A. Rapoport,et al.  Coordination, “Magic,” and Reinforcement Learning in a Market Entry Game , 1998 .

[20]  Werner Güth,et al.  Equilibrium Point Selection in a Class of Market Entry Games , 1982 .

[21]  E. Thorndike.  “Animal Intelligence” , 1898, Nature.

[22]  J. Doyne Farmer,et al.  Physicists attempt to scale the ivory towers of finance , 1999, Comput. Sci. Eng..

[23]  Daniel Friedman,et al.  Individual Learning in Normal Form Games: Some Laboratory Results☆☆☆ , 1997 .

[24]  I. Erev,et al.  Games and Human Behavior , 1999 .

[25]  W. Estes.  Toward a Statistical Theory of Learning. , 1994 .

[26]  Colin Camerer,et al.  Experience‐weighted Attraction Learning in Normal Form Games , 1999 .