Self-tuning experience weighted attraction learning in games

Self-tuning experience weighted attraction (EWA) is a one-parameter theory of learning in games. It addresses a criticism that an earlier model (EWA) has too many parameters, by fixing some parameters at plausible values and replacing others with functions of experience so that they no longer need to be estimated. Consequently, it is econometrically simpler than the popular weighted fictitious play and reinforcement learning models. The functions of experience which replace free parameters “self-tune” over time, adjusting in a way that selects a sensible learning rule to capture subjects’ choice dynamics. For instance, the self-tuning EWA model can turn from a weighted fictitious play into an averaging reinforcement learning as subjects equilibrate and learn to ignore inferior foregone payoffs. The theory was tested on seven different games, and compared to the earlier parametric EWA model and a one-parameter stochastic equilibrium theory (QRE). Self-tuning EWA does as well as EWA in predicting behavior in new games, even though it has fewer parameters, and fits reliably better than the QRE equilibrium benchmark.

[1]  Colin F. Camerer,et al.  Functional EWA: A One-parameter Theory of Learning in Games , 2002 .

[2]  A. Roth,et al.  Jumping the Gun: Imperfections and Institutions Related to the Timing of Market Transactions , 1994 .

[3]  John O. Ledyard,et al.  Computer Testbeds: The Dynamics of Groves-Ledyard Mechanisms , 2003 .

[4]  R. Selten,et al.  End behavior in sequences of finite prisoner's dilemma supergames , 1986 .

[5]  John H. Kagel,et al.  Learning and transfer in signaling games , 2008 .

[6]  Colin Camerer,et al.  Sophisticated EWA Learning and Strategic Teaching in Repeated Games , 2000 .

[7]  Colin Camerer,et al.  A Cognitive Hierarchy Model of Games , 2004 .

[8]  Colin Camerer,et al.  Experience‐weighted Attraction Learning in Normal Form Games , 1999 .

[9]  Frederick W. Rankin,et al.  Selection dynamics and adaptive behavior without much information , 2007 .

[10]  Dilip Mookherjee,et al.  Learning behavior in an experimental matching pennies game , 1994 .

[11]  A. Cabrales,et al.  Equilibrium selection through incomplete information in coordination games: an experimental study , 2007 .

[12]  Teck-Hua Ho,et al.  A learning-based model of repeated games with incomplete information , 2005, Games Econ. Behav..

[13]  D. Stahl,et al.  On Players' Models of Other Players: Theory and Experimental Evidence , 1995 .

[14]  Teck-Hua Ho,et al.  A Functional Theory of Learning in Games , 2001 .

[15]  John B. Van Huyck,et al.  Adaptive behavior and coordination failure , 1997 .

[16]  Alvin E. Roth,et al.  The effect of adding a constant to all payoffs: experimental investigation, and implications for reinforcement learning models , 1999 .

[17]  Alvin E. Roth,et al.  The Effect of Adding a Constant to All Payoffs: Experimental Investigation, and a Reinforcement Learning Model with Self-Adjusting Speed of Learning , 1999 .

[18]  Daniel Friedman,et al.  Individual Learning in Normal Form Games: Some Laboratory Results☆☆☆ , 1997 .

[19]  Charles A. Holt,et al.  Anomalous Behavior in a Traveler's Dilemma? , 1999 .

[20]  E. Hopkins.  Two Competing Models of How People Learn in Games (first version) , 1999 .

[21]  C. Harley.  Learning the evolutionarily stable strategy. , 1981, Journal of theoretical biology.

[22]  R. Sarin,et al.  Strategy Similarity and Coordination , 2004 .

[23]  Charles A. Holt,et al.  Ten Little Treasures of Game Theory and Ten Intuitive Contradictions , 2001 .

[24]  R. McKelvey,et al.  Quantal Response Equilibria for Normal Form Games , 1995 .

[25]  Teck-Hua Ho,et al.  A Parsimonious Model of Stockkeeping-Unit Choice , 2003 .

[26]  Miguel A. Costa-Gomes,et al.  Cognition and Behavior in Normal-Form Games: An Experimental Study , 1998 .

[27]  Dana Heller,et al.  Parametric Adaptive Learning (Draft) , 2000 .

[28]  J. Kagel,et al.  Winner’s Curse , 2014 .

[29]  Colin Camerer.  Behavioral Game Theory: Experiments in Strategic Interaction , 2003 .

[30]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[31]  W. Arthur.  Designing Economic Agents that Act Like Human Agents: A Behavioral Approach to Bounded Rationality , 1991 .

[32]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[33]  Xin Wang,et al.  Individual Differences in EWA Learning with Partial Payoff Information , 2008 .

[34]  A. Rapoport,et al.  Coordination, “Magic,” and Reinforcement Learning in a Market Entry Game , 1998 .

[35]  Dilip Mookherjee,et al.  Learning and Decision Costs in Experimental Constant Sum Games , 1997 .

[36]  Dale O. Stahl,et al.  Sophisticated Learning and Learning Sophistication , 2003 .

[37]  A. Schotter,et al.  An Experimental Study of Belief Learning Using Elicited Beliefs , 2002 .

[38]  T. Sargent.  The Conquest of American Inflation , 1999 .

[39]  Ali Hortaçsu,et al.  Winner's Curse, Reserve Prices and Endogenous Entry: Empirical Insights from Ebay Auctions , 2003 .

[40]  Colin Camerer,et al.  Iterated Dominance and Iterated Best-Response in Experimental P-Beauty Contests , 1998 .

[41]  A. Roth,et al.  Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term* , 1995 .

[42]  Wagner A. Kamakura,et al.  A Parsimonious Model of SKU Choice: Familiarity-based Reinforcement and Response Sensitivity , 1999 .

[43]  A. Marcet,et al.  Recurrent Hyperinflations and Learning , 2003 .

[44]  F. Huddle.  Coordination , 1966, Open Knowledge Institutions.

[45]  M. Kocher,et al.  The Decision Maker Matters: Individual Versus Group Behaviour in Experimental Beauty-Contest Games , 2005 .

[46]  Farshid Vahid,et al.  Predicting How People Play Games: A Simple Dynamic Model of Choice , 2001, Games Econ. Behav..

[47]  S. Hart,et al.  A General Class of Adaptive Strategies , 1999 .

[48]  Teck-Hua Ho,et al.  Sophisticated Experience-Weighted Attraction Learning and Strategic Teaching in Repeated Games , 2002, J. Econ. Theory.

[49]  Guillaume R. Frechette,et al.  Learning in a multilateral bargaining experiment , 2009 .

[50]  Allan Timmermann,et al.  How Learning in Financial Markets Generates Excess Volatility and Predictability in Stock Prices , 1993 .

[51]  A. Rapoport,et al.  Mixed strategies and iterative elimination of strongly dominated strategies: an experimental investi , 2000 .

[52]  Dale O. Stahl,et al.  Rule Learning in Symmetric Normal-Form Games: Theory and Evidence , 2000, Games Econ. Behav..

[53]  Nathaniel T Wilcox,et al.  Heterogeneity and Learning Principles , 2003 .

[54]  Teck-Hua Ho,et al.  Models of Thinking, Learning, and Teaching in Games , 2003 .

[55]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[56]  Patrick H. McAllister,et al.  Adaptive approaches to stochastic programming , 1991, Ann. Oper. Res..

[57]  Jens Josephson,et al.  A numerical analysis of the evolutionary stability of learning rules , 2008 .

[58]  J. Huyck,et al.  Tacit Coordination Games, Strategic Uncertainty, and Coordination Failure , 1990 .

[59]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[60]  Dana Heller.  Parametric Adaptive Learning , 2000 .