Agent Learning Representation - Advice in Modelling Economic Learning

This paper presents an overview on the existing learning models in the economic literature. Furthermore, it discusses which of these models should be used under what circumstances and how adequate learning models can be chosen in simulation approaches. It gives advice for getting along with the many models existing and picking the right one for the own application.

[1]  A. Schotter,et al.  An Experimental Study of Belief Learning Using Elicited Beliefs , 2002 .

[2]  Christian M. Ernst,et al.  Multi-armed Bandit Allocation Indices , 1989 .

[3]  L. Samuelson.  Stochastic Stability in Games with Alternative Best Replies , 1994 .

[4]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[5]  Thomas Brenner,et al.  The Use of Simulations in Developing , 2003 .

[6]  George Yin,et al.  On H -valued Robbins-Monro processes , 1990 .

[7]  W. Estes.  Toward a Statistical Theory of Learning. , 1994 .

[8]  Tilman Börgers.  ON THE RELEVANCE OF LEARNING AND EVOLUTION TO ECONOMIC THEORY , 1996 .

[9]  Philip N. Johnson-Laird,et al.  Naive causality: a mental model theory of causal meaning and reasoning1 , 2001, Cogn. Sci..

[10]  H. Kunzi,et al.  Lectu re Notes in Economics and Mathematical Systems , 1975 .

[11]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[12]  R. M. Elliott,et al.  Behavior of Organisms , 1991 .

[13]  Thomas Brenner,et al.  Noisy decision makers : on errors, noise and inconsistencies in economic behavior , 2001 .

[14]  Jasmina Arifovic.  Genetic algorithm learning and the cobweb model , 1994 .

[15]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[16]  Hans Haller,et al.  Naive Bayesian learning in 2 x 2 matrix games , 1993 .

[17]  Jonathan Bendor,et al.  Aspiration-Based Reinforcement Learning in Repeated Games: An Overview , 2001 .

[18]  John Duffy,et al.  Does observation of others affect learning in strategic environments? An experimental study , 1999, Int. J. Game Theory.

[19]  Herbert Dawid,et al.  Learning of equilibria by a population with minimal information , 1997 .

[20]  Thomas Brenner,et al.  Empirical Calibration of Simulation Models , 2004 .

[21]  A. Shaked,et al.  Altruists, Egoists and Hooligans in a Local Interaction Model , 1996 .

[22]  David M. Messick,et al.  Frontiers in Social Dilemmas Research , 1996 .

[23]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[24]  Brit Grosskopf.  Reinforcement and Directional Learning in the Ultimatum Game with Responder Competition , 2003 .

[25]  Huw David Dixon,et al.  Keeping Up With the Joneses: Competition and the Evolution of Collusion in an Oligopolistic Economy , 1998 .

[26]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[27]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[28]  R. Herrnstein,et al.  Melioration: A Theory of Distributed Choice , 1991 .

[29]  M. Eigen.  Selforganization of matter and the evolution of biological macromolecules , 1971, Naturwissenschaften.

[30]  Thomas Brenner,et al.  Implications of routine-based learning for decision making , 2001 .

[31]  Theresa K. Lant,et al.  Aspiration Level Adaptation: An Empirical Exploration , 1992 .

[32]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[33]  Dale O. Stahl,et al.  Aspiration-based and reciprocity-based rules in learning dynamics for symmetric normal-form games , 2002 .

[34]  W. Arthur.  Designing Economic Agents that Act Like Human Agents: A Behavioral Approach to Bounded Rationality , 1991 .

[35]  C. R. Hallpike,et al.  The principles of social evolution , 1988 .

[36]  E. Kandel,et al.  Molecular biology of learning: modulation of transmitter release. , 1982, Science.

[37]  G. Hodgson.  Evolutionary and Neo-Schumpeterian Approaches to Economics , 1995 .

[38]  A. W. Tucker,et al.  Advances in game theory , 1964 .

[39]  Nicolaas J. Vriend,et al.  An Illustration of the Essential Difference between Individual and Social Learning, and its Consequences for Computational Analyses , 1998 .

[40]  D. North,et al.  Elements of Reason: Shared Mental Models: Ideologies and Institutions , 1994 .

[41]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[42]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[43]  John Duffy,et al.  Agent-Based Models and Human Subject Experiments , 2004 .

[44]  D. Stahl.  Is step-j thinking an arbitrary modelling restriction or a fact of human nature? , 1998 .

[45]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[46]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[47]  J. Jordan,et al.  Bayesian learning in normal form games , 1991 .

[48]  R. Selten.  Features of experimentally observed bounded rationality , 1998 .

[49]  J. Cross.  A theory of adaptive economic behavior , 1983 .

[50]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[51]  N. Kiefer,et al.  Controlling a Stochastic Process with Unknown Parameters , 1988 .

[52]  A. Luchins.  Mechanization in problem solving: The effect of Einstellung. , 1942 .

[53]  Jonathan Bendor,et al.  Aspiration-Based Reinforcement Learning in Repeated Interaction Games: an Overview , 2001, IGTR.

[54]  Tilman Börgers,et al.  Naive Reinforcement Learning With Endogenous Aspirations , 2000 .

[55]  D. Bergemann,et al.  Learning and Strategic Pricing , 1996 .

[56]  Pietro Terna,et al.  Neural Networks for Economic and Financial Modelling , 1995 .

[57]  R. Day.  Profits, Learning and the Convergence of Satisficing to Marginalism , 1967 .

[58]  Debraj Ray,et al.  Evolving Aspirations and Cooperation , 1998 .

[59]  R. Herrnstein.  On the law of effect. , 1970, Journal of the experimental analysis of behavior.

[60]  A. Zellner.  An Introduction to Bayesian Inference in Econometrics , 1971 .

[61]  H. Young,et al.  The Evolution of Conventions , 1993 .

[62]  R. Selten,et al.  Bounded rationality: The adaptive toolbox , 2000 .

[63]  D. Stahl,et al.  Experimental evidence on players' models of other players , 1994 .

[64]  Yngve Ramstad.  On the Nature of Economic Evolution: John R. Commons and the Metaphor of Artificial Selection , 1994 .

[65]  A. Roth,et al.  Learning in Extensive-Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term* , 1995 .

[66]  John R. Anderson.  Cognitive Psychology and Its Implications , 1980 .

[67]  R. Selten,et al.  End behavior in sequences of finite prisoner's dilemma supergames , 1986 .

[68]  K. Schlag.  Why Imitate, and If So, How?, : A Boundedly Rational Approach to Multi-armed Bandits , 1998 .

[69]  Nicolaas J. Vriend,et al.  Imitation of successful behaviour in cournot markets , 2003 .

[70]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[71]  P. Johnson-Laird.  Mental models , 1989 .

[72]  Thomas Brenner,et al.  Can evolutionary algorithms describe learning processes? , 1998 .

[73]  H. Kelley,et al.  The social psychology of groups , 1960 .

[74]  Richard H. Day,et al.  How to Co-operate in Business without Really Trying: A Learning Model of Decentralized Decision Making , 1968, Journal of Political Economy.

[75]  Karl H. Schlag,et al.  Which One Should I Imitate , 1999 .

[76]  Fernando Vega-Redondo,et al.  Convergence of aspirations and (partial) cooperation in the prisoner's dilemma , 1999, Int. J. Game Theory.

[77]  M. Bray.  Learning, estimation, and the stability of rational expectations , 1982 .

[78]  Jörg Oechssler,et al.  Stability of the Cournot process – experimental evidence , 2002, Int. J. Game Theory.

[79]  E. Kalai,et al.  Rational Learning Leads to Nash Equilibrium , 1993 .

[80]  Amit Pazgal.  Satisficing leads to cooperation in mutual interests games , 1998 .

[81]  Frederick Mosteller,et al.  Stochastic Models for Learning , 1956 .

[82]  R. Rob,et al.  Learning, Mutation, and Long Run Equilibria in Games , 1993 .

[83]  Ya-Ru Chen,et al.  Aspiration-Level Adaptation in an American Financial Services Organization: A Field Study , 2002, Manag. Sci..

[84]  Herbert Dawid,et al.  Adaptive Learning by Genetic Algorithms , 1996 .

[85]  Thomas Riechmann,et al.  Learning and behavioral stability An economic interpretation of genetic algorithms , 1999 .

[86]  Michael Waldman,et al.  The role of altruism in economic interaction , 1993 .

[87]  T. Brenner,et al.  Melioration learning in games with constant and frequency-dependent pay-offs , 2003 .

[88]  U. Witt.  Evolutionary Economics: An interpretative survey , 2001 .

[89]  J. Jordan.  Bayesian Learning in Repeated Games , 1995 .

[90]  L. Shapley.  SOME TOPICS IN TWO-PERSON GAMES , 1963 .

[91]  N. Mackintosh.  Pavlov and Associationism , 2003, The Spanish Journal of Psychology.

[92]  Dale O. Stahl,et al.  Rule Learning in Symmetric Normal-Form Games: Theory and Evidence , 2000, Games Econ. Behav..

[93]  U. Witt.  special issue: Learning to consume - A theory of wants and the growth of demand , 2001 .

[94]  L. Festinger,et al.  Wish, expectation, and group standards as factors influencing level of aspiration. , 1942 .

[95]  Leigh Tesfatsion,et al.  Introduction to the CE Special Issue on Agent-Based Computational Economics , 2001 .

[96]  Shu-Heng Chen,et al.  Equilibrium Selection via Adaptation: Using Genetic Programming to Model Learning in a Coordination Game , 2005 .

[97]  I. Gilboa,et al.  Case-Based Decision Theory , 1995 .

[98]  H. Bekkering,et al.  To imitate or not to imitate? How the brain can do it, that is the question! , 2003, Brain and Cognition.

[99]  D. Kahneman.  Maps of Bounded Rationality: Psychology for Behavioral Economics , 2003 .

[100]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[101]  Alvin E. Roth,et al.  The effect of adding a constant to all payoffs: experimental investigation, and implications for reinforcement learning models , 1999 .

[102]  John O. Ledyard,et al.  Scaling Up Learning Models in Public Good Games , 2004 .

[103]  Nick Feltovich,et al.  Reinforcement-based vs. Belief-based Learning Models in Experimental Asymmetric-information Games , 2000 .

[104]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .

[105]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[106]  R. Rescorla.  A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .

[107]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[108]  J. Bruner.  Beyond the Information Given: Studies in the Psychology of Knowing , 1973 .

[109]  Kristian Lindgren,et al.  Evolutionary phenomena in simple dynamics , 1992 .

[110]  E. Thorndike.  The fundamentals of learning , 1972 .

[111]  Antonio J. Morales.  Absolutely expedient imitative behavior , 2003, Int. J. Game Theory.

[112]  U. Witt.  Learning to consume – A theory of wants and the growth of demand , 2001 .

[113]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[114]  Colin Camerer,et al.  Advances in behavioral economics , 2004 .

[115]  Learning in a large square economy , 1994 .

[116]  John Maynard Smith,et al.  Evolution and the Theory of Games: Preface , 1982 .

[117]  Fang-Fang Tang,et al.  A comparative study on learning in a normal form game experiment , 2003 .

[118]  T. Brenner.  Modelling Learning in Economics , 1999 .

[119]  B. Edmonds.  Modelling Bounded Rationality in Agent-Based Simulations Using the Evolution of Mental Models , 1999 .

[120]  Maik Heinemann,et al.  Adaptive learning of rational expectations using neural networks , 2000 .

[121]  Eyal Winter,et al.  An experimental study of coordination and learning in iterated two-market entry games , 2000 .

[122]  R. Hegselmann.  Social Dilemmas in Lineland and Flatland , 1996 .

[123]  Shu-Heng Chen,et al.  Evolving traders and the business school with genetic programming: A new architecture of the agent-based artificial stock market , 2001 .

[124]  Colin Camerer,et al.  Experience‐weighted Attraction Learning in Normal Form Games , 1999 .

[125]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[126]  J. Holland,et al.  Artificial Adaptive Agents in Economic Theory , 1991 .

[127]  E. Fehr.  A Theory of Fairness, Competition and Cooperation , 1998 .

[128]  F. Klix,et al.  Bauplan für eine Seele. , 2001 .

[129]  M. Nowak,et al.  THE SPATIAL DILEMMAS OF EVOLUTION , 1993 .

[130]  Daniel Friedman,et al.  Individual Learning in Normal Form Games: Some Laboratory Results☆☆☆ , 1997 .

[131]  D. Fogel.  Evolutionary algorithms in theory and practice , 1997, Complex..

[132]  T. Brenner,et al.  On the Behavior of Proposers in Ultimatum Games , 2003 .

[133]  Siegfried K. Berninghaus,et al.  Time horizon and equilibrium selection in tacit coordination games: Experimental results , 1998 .

[134]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[135]  Reaction to Price Changes and Aspiration Level Adjustments , 1994 .

[136]  Ulrich Witt,et al.  Firms' market behavior under imperfect information and economic natural selection , 1986 .

[137]  F. Vega-Redondo.  The evolution of Walrasian behavior , 1997 .

[138]  V. Crawford.  Adaptive dynamics in coordination games , 1995 .

[139]  AMIT PAZGAL.  Satisficing leads to cooperation in mutual interests games , 1997, Int. J. Game Theory.

[140]  Colin F. Camerer,et al.  Experience-Weighted Attraction Learning in Sender-Receiver Signaling Games , 2001 .

[141]  D. Kahneman.  Maps of Bounded Rationality , 2002 .

[142]  T. Sargent,et al.  Convergence of Least Squares Learning Mechanisms in Self- Referential Linear Stochastic Models* , 1989 .

[143]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[144]  J. Cross.  A Stochastic Learning Model of Economic Behavior , 1973 .

[145]  R. Sarin,et al.  Payoff Assessments without Probabilities: A Simple Dynamic Model of Choice , 1999 .

[146]  Oliver Kirchkamp,et al.  Spatial Evolution of Automata in the Prisoner's Dilemma , 2000, Social Science Microsimulation.

[147]  Stephen J. Mezias,et al.  Aspiration level effects: An empirical investigation , 1988 .

[148]  Glenn Ellison.  Learning, Local Interaction, and Coordination , 1993 .

[149]  Mario Calderini,et al.  Compound Learning, Neural Nets And The Competitive Process , 1998 .

[150]  L. Marengo,et al.  A learning-to-forecast experiment on the foreign exchange market with a classifier system , 1997 .