Aspiration dynamics generate robust predictions in structured populations

Evolutionary game dynamics in structured populations are strongly affected by updating rules. Previous studies usually focus on imitation-based rules, which rely on payoff information of social peers. Recent behavioral experiments suggest that whether individuals use such social information for strategy updating may be crucial to the outcomes of social interactions. This hints at the importance of considering updating rules without dependence on social peers' payoff information, which, however, is rarely investigated. Here, we study aspiration-based self-evaluation rules, with which individuals self-assess the performance of strategies by comparing own payoffs with an imaginary value they aspire, called the aspiration level. We explore the fate of strategies on population structures represented by graphs or networks. Under weak selection, we analytically derive the condition for strategy dominance, which is found to coincide with the classical condition of risk-dominance. This condition holds for all networks and all distributions of aspiration levels, and for individualized ways of self-evaluation. Our condition can be intuitively interpreted: one strategy prevails over the other if the strategy brings more satisfaction to individuals than the other does. Our work thus sheds light on the intrinsic difference between evolutionary dynamics induced by aspiration-based and imitation-based rules.

[1]  Debraj Ray,et al.  Evolving Aspirations and Cooperation , 1998 .

[2]  K. Sigmund,et al.  The efficiency of adapting aspiration levels , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[3]  Rick Durrett,et al.  Extrapolating weak selection in evolutionary games , 2018, bioRxiv.

[4]  Naoki Masuda,et al.  Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin , 2016, PLoS Comput. Biol..

[5]  Lei Zhou,et al.  Supplementary Methods: Individualised aspiration dynamics: Calculation by proofs , 2018 .

[6]  H. Simon,et al.  Theories of Decision-Making in Economics and Behavioural Science , 1966 .

[7]  H. Ohtsuki,et al.  Strategy selection in structured populations. , 2009, Journal of theoretical biology.

[8]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[9]  Martin A Nowak,et al.  Multiple strategies in structured populations , 2011, Proceedings of the National Academy of Sciences.

[10]  Ángel Sánchez,et al.  Evolutionary game theory: Temporal and spatial effects beyond replicator dynamics , 2009, Physics of life reviews.

[11]  Ángel Sánchez,et al.  Promotion of cooperation on networks? The myopic best response case , 2009, ArXiv.

[12]  Lourdes Araujo,et al.  Social Experiments in the Mesoscale: Humans Playing a Spatial Prisoner's Dilemma , 2010, PloS one.

[13]  Martin A. Nowak,et al.  Spatial structure often inhibits the evolution of cooperation in the snowdrift game , 2022 .

[14]  Arne Traulsen,et al.  Pairwise comparison and selection temperature in evolutionary game dynamics. , 2007, Journal of theoretical biology.

[15]  David G. Rand,et al.  Static network structure can stabilize human cooperation , 2014, Proceedings of the National Academy of Sciences.

[16]  M. Nowak,et al.  Evolutionary dynamics in structured populations , 2010, Philosophical Transactions of the Royal Society B: Biological Sciences.

[17]  Martin A. Nowak,et al.  Evolutionary dynamics on any population structure , 2016, Nature.

[18]  F. C. Santos,et al.  Evolutionary dynamics of social dilemmas in structured heterogeneous populations. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[19]  D. Fudenberg,et al.  Emergence of cooperation and evolutionary stability in finite populations , 2004, Nature.

[20]  Long Wang,et al.  Aspiration dynamics in structured population acts as if in a well-mixed one , 2015, Scientific Reports.

[21]  Tilman Börgers,et al.  Learning Through Reinforcement and Replicator Dynamics , 1997 .

[22]  David A. Siegel,et al.  A Behavioral Theory of Elections , 2011 .

[23]  H. Ohtsuki,et al.  A simple rule for the evolution of cooperation on graphs and social networks , 2006, Nature.

[24]  Franz J Weissing,et al.  Consistent individual differences in human social learning strategies , 2014, Nature Communications.

[25]  Long Wang,et al.  Promotion of cooperation induced by appropriate payoff aspirations in a small-world networked game. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Angel Sánchez,et al.  A comparative analysis of spatial Prisoner's Dilemma experiments: Conditional cooperation and payoff irrelevance , 2014, Scientific Reports.

[27]  Friederike Mengel,et al.  Learning in Network Games , 2012 .

[28]  J. Cuesta,et al.  Heterogeneous networks do not promote cooperation when humans play a Prisoner’s Dilemma , 2012, Proceedings of the National Academy of Sciences.

[29]  U. Fischbacher,et al.  Are People Conditionally Cooperative? Evidence from a Public Goods Experiment , 2001 .

[30]  H. Ohtsuki,et al.  Mutation-selection equilibrium in games with multiple strategies. , 2008, Journal of theoretical biology.

[31]  Corina E Tarnita,et al.  Measures of Relative Fitness of Social Behaviors in Finite Structured Population Models , 2014, The American Naturalist.

[32]  Franz J Weissing,et al.  Focus on the success of others leads to selfish behavior , 2015, Proceedings of the National Academy of Sciences.

[33]  Naoki Masuda,et al.  Reinforcement learning account of network reciprocity , 2017, PloS one.

[34]  Dirk Helbing,et al.  Emergence of social cohesion in a model society of greedy, mobile individuals , 2011, Proceedings of the National Academy of Sciences.

[35]  Long Wang,et al.  Aspiration dynamics of multi-player games in finite populations , 2014, Journal of The Royal Society Interface.

[36]  Long Wang,et al.  Universality of weak selection. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[37]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[38]  M. Nowak,et al.  Stochastic dynamics of invasion and fixation. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  M. Nowak,et al.  Evolutionary games and spatial chaos , 1992, Nature.

[40]  C. Hauert,et al.  Social evolution in structured populations , 2014, Nature Communications.