Evolutionary game theory: Temporal and spatial effects beyond replicator dynamics

Evolutionary game dynamics is one of the most fruitful frameworks for studying evolution in different disciplines, from Biology to Economics. Within this context, the approach of choice for many researchers is the so-called replicator equation, that describes mathematically the idea that those individuals performing better have more offspring and thus their frequency in the population grows. While very many interesting results have been obtained with this equation in the three decades elapsed since it was first proposed, it is important to realize the limits of its applicability. One particularly relevant issue in this respect is that of non-mean-field effects, that may arise from temporal fluctuations or from spatial correlations, both neglected in the replicator equation. This review discusses these temporal and spatial effects focusing on the non-trivial modifications they induce when compared to the outcome of replicator dynamics. Alongside this question, the hypothesis of linearity and its relation to the choice of the rule for strategy update is also analyzed. The discussion is presented in terms of the emergence of cooperation, as one of the current key problems in Biology and in other disciplines.

[1]  George Price’s,et al.  George Price ’ s Contributions to Evolutionary Genetics , 1995 .

[2]  G. Szabó,et al.  Evolutionary games on graphs , 2006, cond-mat/0607344.

[3]  L. Peliti Fitness Landscapes and Evolution , 1995, cond-mat/9505003.

[4]  M. Doebeli,et al.  Spatial evolutionary game theory: Hawks and Doves revisited , 1996, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[5]  Thilo Gross,et al.  Adaptive coevolutionary networks: a review , 2007, Journal of The Royal Society Interface.

[6]  Yi-Cheng Zhang,et al.  Emergence of cooperation and organization in an evolutionary game , 1997 .

[7]  Michael Doebeli,et al.  Spatial structure often inhibits the evolution of cooperation in the snowdrift game , 2004, Nature.

[8]  Angel Sánchez,et al.  Mesoscopic Structure Conditions the Emergence of Cooperation on Social Networks , 2006, PloS one.

[9]  Robert G. Gallager,et al.  Discrete Stochastic Processes , 1995 .

[10]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[11]  U. Fischbacher,et al.  Strong reciprocity, human cooperation, and the enforcement of social norms , 2002, Human nature.

[12]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[13]  Jacek Miekisz,et al.  Equilibrium selection in evolutionary games with random matching of players. , 2004, Journal of theoretical biology.

[14]  R. Sugden The Economics of Rights, Co-Operation, and Welfare , 1986 .

[15]  Petter Holme,et al.  Prisoners' dilemma in real-world acquaintance networks: spikes and quasiequilibria induced by the interplay between structure and dynamics. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  R. Rob,et al.  Learning, Mutation, and Long Run Equilibria in Games , 1993 .

[17]  J. Roughgarden Theory of Population Genetics and Evolutionary Ecology: An Introduction , 1995 .

[18]  M N,et al.  The Evolution of Cooperation in a Lattice-Structured Population , 1996 .

[19]  David B. Fogel,et al.  On the instability of evolutionary stable strategies in small populations , 1998 .

[20]  Gergely Boza,et al.  Asynchronous snowdrift game with synergistic effect as a model of cooperation , 2006 .

[21]  M. Nowak,et al.  A generalized adaptive dynamics framework can describe the evolutionary Ultimatum Game. , 2001, Journal of theoretical biology.

[22]  Ángel Sánchez,et al.  Effect of spatial structure on the evolution of cooperation , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  S. Ellner,et al.  Rapid evolution drives ecological dynamics in a predator–prey system , 2003, Nature.

[24]  J. J. Arenzon,et al.  Random mobility and spatial structure often enhance cooperation. , 2008, Journal of theoretical biology.

[25]  J Gómez-Gardeñes,et al.  Dynamical organization of cooperation in complex topologies. , 2007, Physical review letters.

[26]  S. Gächter Behavioral Game Theory , 2008, Encyclopedia of Evolutionary Psychological Science.

[27]  J. Gómez-Gardeñes,et al.  Robustness of cooperation in the evolutionary prisoner's dilemma on complex networks , 2007, q-bio/0703019.

[28]  C. Hauert,et al.  Coevolutionary dynamics: from finite to infinite populations. , 2004, Physical review letters.

[29]  H. Peyton Young,et al.  Stochastic Evolutionary Game Dynamics , 1990 .

[30]  Richard T. Boylan Continuous Approximation of Dynamical Systems with Randomly Matched Individuals , 1995 .

[31]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[32]  J. Hofbauer,et al.  Evolutionary game dynamics , 2011 .

[33]  G T Vickers,et al.  The spatial struggle of tit-for-tat and defect. , 1995, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[34]  L. Blume The Statistical Mechanics of Strategic Interaction , 1993 .

[35]  Drew Fudenberg,et al.  Evolutionary game dynamics in finite populations , 2004, Bulletin of mathematical biology.

[36]  A. Hendry,et al.  PERSPECTIVE: THE PACE OF MODERN LIFE: MEASURING RATES OF CONTEMPORARY MICROEVOLUTION , 1999, Evolution; international journal of organic evolution.

[37]  F. C. Santos,et al.  Scale-free networks provide a unifying framework for the emergence of cooperation. , 2005, Physical review letters.

[38]  Kan Chen,et al.  Scale-invariant behavior in a spatial game of prisoners' dilemma. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  M. Nowak,et al.  Stochastic dynamics of invasion and fixation. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[40]  B A Huberman,et al.  Evolutionary games and computer simulations. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[41]  P. Moran,et al.  The statistical processes of evolutionary theory. , 1963 .

[42]  P. Taylor,et al.  Evolutionarily Stable Strategies and Game Dynamics , 1978 .

[43]  Eörs Szathmáry,et al.  The Major Transitions in Evolution , 1997 .

[44]  V. Eguíluz,et al.  Growing scale-free networks with small-world behavior. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[45]  Anatol Rapoport,et al.  A taxonomy of 2x2 games , 1966 .

[46]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[47]  Francisco C. Santos,et al.  Cooperation Prevails When Individuals Adjust Their Social Ties , 2006, PLoS Comput. Biol..

[48]  A. Licht Games Commissions Play: 2x2 Games of International Securities Regulation , 1997 .

[49]  Wang Ying-Hai,et al.  Prisoner's Dilemma Game with Heterogeneous Influential Effect on Regular Small-World Networks , 2006 .

[50]  A. Bennett The Origin of Species by means of Natural Selection; or the Preservation of Favoured Races in the Struggle for Life , 1872, Nature.

[51]  John N. Tsitsiklis,et al.  Introduction to Probability , 2002 .

[52]  Elizabeth Pennisi,et al.  How Did Cooperative Behavior Evolve? , 2005, Science.

[53]  M. Perc,et al.  Coevolution of teaching activity promotes cooperation , 2008, 0803.4091.

[54]  Yamir Moreno,et al.  Complex Cooperative Networks from Evolutionary Preferential Attachment , 2008, PloS one.

[55]  John C. Harsanyi,et al.  Общая теория выбора равновесия в играх / A General Theory of Equilibrium Selection in Games , 1989 .

[56]  Oliver Kirchkamp,et al.  Simultaneous Evolution of Learning Rules and Strategies , 1996 .

[57]  Tian Qiu,et al.  Effects of dynamic response time in an evolving market , 2009 .

[58]  H. Hinrichsen Non-equilibrium critical phenomena and phase transitions into absorbing states , 2000, cond-mat/0001070.

[59]  F. Mengel,et al.  Cooperation Through Imitation and Exclusion In Networks , 2011 .

[60]  M. Tomassini,et al.  Hawks and Doves on small-world networks. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[61]  C. Hauert,et al.  Game theory and physics , 2005 .

[62]  J. Kingman A FIRST COURSE IN STOCHASTIC PROCESSES , 1967 .

[63]  R. Trivers The Evolution of Reciprocal Altruism , 1971, The Quarterly Review of Biology.

[64]  Carlos P. Roca,et al.  The importance of selection rate in the evolution of cooperation , 2005, q-bio/0512045.

[65]  Lawrence E. Blume,et al.  How noise matters , 2003, Games Econ. Behav..

[66]  M. Perc,et al.  Promoting cooperation in social dilemmas via simple coevolutionary rules , 2008, 0812.1122.

[67]  Oliver Kirchkamp,et al.  Spatial Evolution of Automata in the Prisoner's Dilemma , 2000, Social Science Microsimulation.

[68]  F. C. Santos,et al.  Graph topology plays a determinant role in the evolution of cooperation , 2006, Proceedings of the Royal Society B: Biological Sciences.

[69]  A. W. F. Edwards,et al.  The statistical processes of evolutionary theory , 1963 .

[70]  Zhi-Xi Wu,et al.  Spatial prisoner's dilemma game with volunteering in Newman-Watts small-world networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[71]  P. Kollock SOCIAL DILEMMAS: The Anatomy of Cooperation , 1998 .

[72]  Arne Traulsen,et al.  Coevolution of strategy and structure in complex networks with dynamical linking. , 2006, Physical review letters.

[73]  M A Nowak,et al.  Spatial games and the maintenance of cooperation. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[74]  Wen-Xu Wang,et al.  Effects of average degree on cooperation in networked evolutionary game , 2006 .

[75]  M. Nowak Evolutionary Dynamics: Exploring the Equations of Life , 2006 .

[76]  W. Hamilton The genetical evolution of social behaviour. II. , 1964, Journal of theoretical biology.

[77]  W. Hamilton The genetical evolution of social behaviour. I. , 1964, Journal of theoretical biology.

[78]  Dirk Helbing Interrelations between stochastic equations for systems with pair interactions , 1992 .

[79]  Arne Traulsen,et al.  Repeated games and direct reciprocity under active linking. , 2008, Journal of theoretical biology.

[80]  K. Schlag Why Imitate, and If So, How?, : A Boundedly Rational Approach to Multi-armed Bandits , 1998 .

[81]  Marco Tomassini,et al.  Evolutionary games on networks and payoff invariance under replicator dynamics , 2009, Biosyst..

[82]  Angel Sánchez,et al.  Evolving learning rules and emergence of cooperation in spatial prisoner's dilemma. , 2008, Journal of theoretical biology.

[83]  G. Szabó,et al.  Cooperation in the noisy case: Prisoner's dilemma game on two types of regular random graphs. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[84]  H. Gintis Strong reciprocity and human sociality. , 2000, Journal of theoretical biology.

[85]  M. Kuperman,et al.  Social games in a social network. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[86]  D. Fogel,et al.  On the instability of evolutionary stable strategies. , 1997, Bio Systems.

[87]  M H Vainstein,et al.  Disordered environments in spatial games. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[88]  A. Traulsen,et al.  Non-Gaussian fluctuations arising from finite populations: Exact results for the evolutionary Moran process. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[89]  P. Grim,et al.  Spatialization and greater generosity in the stochastic Prisoner's Dilemma. , 1996, Bio Systems.

[90]  Robert van Rooij,et al.  The Stag Hunt and the Evolution of Social Structure , 2007, Stud Logica.

[91]  J. Chiappin,et al.  Emergence of cooperation among interacting individuals. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[92]  A Díaz-Guilera,et al.  Self-similar community structure in a network of human interactions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[93]  Ángel Sánchez,et al.  Promotion of cooperation on networks? The myopic best response case , 2009, ArXiv.

[94]  M. Macy,et al.  Learning dynamics in social dilemmas , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[95]  G. Szabó,et al.  Selection of dynamical rules in spatial Prisoner's Dilemma games , 2009, 0911.0661.

[96]  U. Fischbacher,et al.  The nature of human altruism , 2003, Nature.

[97]  Samuel Karlin,et al.  A First Course on Stochastic Processes , 1968 .

[98]  Glenn Ellison Learning, Local Interaction, and Coordination , 1993 .

[99]  Angel Sánchez,et al.  Altruism may arise from individual selection. , 2004, Journal of theoretical biology.

[100]  R. Fagen,et al.  When Doves Conspire: Evolution of Nondamaging Fighting Tactics in a Nonrandom-Encounter Animal Conflict Model , 1980, The American Naturalist.

[101]  D. Fudenberg,et al.  Emergence of cooperation and evolutionary stability in finite populations , 2004, Nature.

[102]  Mike Mesterton-Gibbons,et al.  Genetic and cultural evolution of cooperation , 2004 .

[103]  Arne Traulsen,et al.  The different limits of weak selection and the evolutionary dynamics of finite populations. , 2007, Journal of theoretical biology.

[104]  Günter Haag,et al.  Economic Evolution and Demographic Change , 1992 .

[105]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[106]  M. Nowak,et al.  Evolutionary games and spatial chaos , 1992, Nature.

[107]  George R. Price,et al.  Selection and Covariance , 1970, Nature.

[108]  T. Quinn,et al.  Rapid evolution of reproductive isolation in the wild: evidence from introduced salmon. , 2000, Science.

[109]  G. Szabó,et al.  Phase diagrams for an evolutionary prisoner's dilemma game on two-dimensional lattices. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[110]  Alessandra Cassar,et al.  Coordination and Cooperation in Local, Random and Small World Networks: Experimental Evidence , 2002, Games Econ. Behav..

[111]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[112]  B. Drossel Biological evolution and statistical physics , 2001, cond-mat/0101409.

[113]  Colin Camerer,et al.  Foundations of Human Sociality - Economic Experiments and Ethnographic: Evidence From Fifteen Small-Scale Societies , 2004 .

[114]  長野 敬 On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life , 2004 .

[115]  S. Assenza,et al.  Enhancement of cooperation in highly clustered scale-free networks. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[116]  Margarita Ifti,et al.  Effects of neighbourhood size and connectivity on the spatial Continuous Prisoner's Dilemma. , 2004, Journal of theoretical biology.

[117]  M. Perc,et al.  Social diversity and promotion of cooperation in the spatial prisoner's dilemma game. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[118]  C. Harley Learning the evolutionarily stable strategy. , 1981, Journal of theoretical biology.

[119]  V. Eguíluz,et al.  Cooperation and the Emergence of Role Differentiation in the Dynamics of Social Networks1 , 2005, American Journal of Sociology.

[120]  K. Lindgren,et al.  Evolutionary dynamics of spatial games , 1994 .

[121]  Arne Traulsen,et al.  Coevolutionary dynamics in large, but finite populations. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[122]  Masaki Tomochi Defectors' niches: prisoner's dilemma game on disordered networks , 2004, Soc. Networks.

[123]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[124]  F. C. Santos,et al.  Evolutionary dynamics of social dilemmas in structured heterogeneous populations. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[125]  Christoph Hauert,et al.  Effects of Space in 2 × 2 Games , 2002, Int. J. Bifurc. Chaos.

[126]  Hilla Peretz,et al.  The , 1966 .

[127]  H. Ohtsuki,et al.  Breaking the symmetry between interaction and replacement in evolutionary dynamics on graphs. , 2007, Physical review letters.

[128]  D. Helbing,et al.  The outbreak of cooperation among success-driven individuals under noisy conditions , 2009, Proceedings of the National Academy of Sciences.

[129]  K. Aihara,et al.  Spatial prisoner's dilemma optimally played in small-world networks , 2003 .

[130]  M. Nowak,et al.  Evolution of indirect reciprocity by image scoring , 1998, Nature.

[131]  Arne Traulsen,et al.  Stochastic payoff evaluation increases the temperature of selection. , 2007, Journal of theoretical biology.

[132]  Peter Secretan Learning , 1965, Mental Health.

[133]  Chen Xu,et al.  Networking effects on cooperation in evolutionary snowdrift game , 2006 .

[134]  J. Cuesta,et al.  Time scales in evolutionary dynamics. , 2006, Physical review letters.

[135]  M. Nowak,et al.  Empathy leads to fairness , 2002, Bulletin of mathematical biology.

[136]  Michael X Cohen,et al.  THE ROLE OF SOCIAL STRUCTURE IN THE MAINTENANCE OF COOPERATIVE REGIMES , 2001 .

[137]  M. Nowak Five Rules for the Evolution of Cooperation , 2006, Science.

[138]  M. Bacharach Economics and the Theory of Games , 2019 .

[139]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[140]  Víctor M Eguíluz,et al.  Coevolution of dynamical states and interactions in dynamic networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[141]  Arne Traulsen,et al.  Stochastic sampling of interaction partners versus deterministic payoff assignment. , 2009, Journal of theoretical biology.

[142]  K. Kaski,et al.  Spatial snowdrift game with myopic agents , 2005 .

[143]  S. Gould,et al.  Exaptation—a Missing Term in the Science of Form , 1982, Paleobiology.

[144]  F. Vega-Redondo,et al.  Efficient Equilibrium Selection in Evolutionary Games with Random Matching , 1996 .

[145]  G. Szabó,et al.  Evolutionary prisoner's dilemma game on a square lattice , 1997, cond-mat/9710096.

[146]  V. Latora,et al.  Complex networks: Structure and dynamics , 2006 .

[147]  Matteo Marsili,et al.  The rise and fall of a networked society: a formal model. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[148]  M. Nowak,et al.  Unifying evolutionary dynamics. , 2002, Journal of theoretical biology.

[149]  H. Ohtsuki,et al.  The replicator equation on graphs. , 2006, Journal of theoretical biology.

[150]  W. Güth,et al.  An experimental analysis of ultimatum bargaining , 1982 .

[151]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[152]  Heinz Mühlenbein,et al.  Evolution of Cooperation in a Spatial Prisoner's Dilemma , 2002, Adv. Complex Syst..

[153]  Brauchli,et al.  Evolution of cooperation in spatially structured populations , 1999, Journal of theoretical biology.

[154]  Peter Hammerstein,et al.  Genetic and cultural evolution of cooperation , 2003 .

[155]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[156]  Constanza Fosco,et al.  Cooperation Through Imitation and Exclusion In Networks , 2009 .

[157]  Richard T. Boylan Laws of large numbers for dynamical systems with randomly matched individuals , 1992 .

[158]  Jens Christian Claussen,et al.  Discrete stochastic processes, replicator and Fokker-Planck equations of coevolutionary dynamics in finite and infinite populations , 2008, ArXiv.

[159]  J. Nash Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.