Memory-n strategies of direct reciprocity

Significance Direct reciprocity is one of the fundamental mechanisms for cooperation. It is based on the idea that individuals are more likely to cooperate if they can expect their beneficiaries to remember and to return their cooperative acts in future. Previous computational models, however, often had to restrict the number of past rounds subjects can memorize. Herein we suggest an alternative approach. We propose general properties that robust cooperative strategies ought to have. Then we characterize all memory-n strategies that meet these properties, and we show that such strategies naturally emerge across different evolutionary scenarios. Our results are applicable to general social dilemmas of arbitrary size. For some dilemmas, longer memory is all it takes for cooperation to evolve. Humans routinely use conditionally cooperative strategies when interacting in repeated social dilemmas. They are more likely to cooperate if others cooperated before, and are ready to retaliate if others defected. To capture the emergence of reciprocity, most previous models consider subjects who can only choose from a restricted set of representative strategies, or who react to the outcome of the very last round only. As players memorize more rounds, the dimension of the strategy space increases exponentially. This increasing computational complexity renders simulations for individuals with higher cognitive abilities infeasible, especially if multiplayer interactions are taken into account. Here, we take an axiomatic approach instead. We propose several properties that a robust cooperative strategy for a repeated multiplayer dilemma should have. These properties naturally lead to a unique class of cooperative strategies, which contains the classical Win–Stay Lose–Shift rule as a special case. A comprehensive numerical analysis for the prisoner’s dilemma and for the public goods game suggests that strategies of this class readily evolve across various memory-n spaces. Our results reveal that successful strategies depend not only on how cooperative others were in the past but also on the respective context of cooperation.

[1]  Joshua B. Plotkin,et al.  Small groups and long memories promote cooperation , 2016, Scientific Reports.

[2]  Martin A Nowak,et al.  Comparing reactive and memory-one strategies of direct reciprocity , 2016, Scientific Reports.

[3]  Julián García,et al.  In and out of equilibrium I: Evolution of strategies in repeated games with discounting , 2007, J. Econ. Theory.

[4]  Yi Tao,et al.  Evolution of Conformity in Social Dilemmas , 2015, PloS one.

[5]  Arne Traulsen,et al.  Partners or rivals? Strategies for the iterated prisoner's dilemma☆ , 2015, Games Econ. Behav..

[6]  Francisco C. Santos,et al.  Evolution of All-or-None Strategies in Repeated Public Goods Dilemmas , 2014, PLoS Comput. Biol..

[7]  Arne Traulsen,et al.  Cooperation and control in multiplayer social dilemmas , 2014, Proceedings of the National Academy of Sciences.

[8]  Angel Sánchez,et al.  A comparative analysis of spatial Prisoner's Dilemma experiments: Conditional cooperation and payoff irrelevance , 2014, Scientific Reports.

[9]  Alexander J. Stewart,et al.  Collapse of cooperation in evolving games , 2014, Proceedings of the National Academy of Sciences.

[10]  C. D. De Dreu Human Cooperation , 2013, Psychological science in the public interest : a journal of the American Psychological Society.

[11]  Alexander J. Stewart,et al.  From extortion to generosity, evolution in the Iterated Prisoner’s Dilemma , 2013, Proceedings of the National Academy of Sciences.

[12]  Sebastian J. Goerg,et al.  Fusing enacted and expected mimicry generates a winning strategy that promotes the evolution of cooperation , 2013, Proceedings of the National Academy of Sciences.

[13]  Martin A Nowak,et al.  Evolution of extortion in Iterated Prisoner’s Dilemma games , 2012, Proceedings of the National Academy of Sciences.

[14]  Ethan Akin,et al.  Stable Cooperative Solutions for the Iterated Prisoner's Dilemma , 2012, ArXiv.

[15]  Alexander J. Stewart,et al.  Extortion and cooperation in the Prisoner’s Dilemma , 2012, Proceedings of the National Academy of Sciences.

[16]  David G. Rand,et al.  Direct reciprocity in structured populations , 2012, Proceedings of the National Academy of Sciences.

[17]  W. Press,et al.  Iterated Prisoner’s Dilemma contains strategies that dominate any evolutionary opponent , 2012, Proceedings of the National Academy of Sciences.

[18]  Luis A. Martinez-Vaquero,et al.  Generosity Pays in the Presence of Direct Reciprocity: A Comprehensive Study of 2×2 Repeated Games , 2012, PloS one.

[19]  F. C. Santos,et al.  Emergence of fairness in repeated group interactions. , 2012, Physical review letters.

[20]  Chaitanya S. Gokhale,et al.  How small are small mutation rates? , 2011, Journal of Mathematical Biology.

[21]  M. Dufwenberg Game theory. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[22]  Manfred Milinski,et al.  The Calculus of Selfishness , 2011 .

[23]  David G. Rand,et al.  Slow to Anger and Fast to Forgive: Cooperation in an Uncertain World , 2010 .

[24]  Arne Traulsen,et al.  Human strategy updating in evolutionary games , 2010, Proceedings of the National Academy of Sciences.

[25]  Martin A. Nowak,et al.  Stochastic evolutionary dynamics of direct reciprocity , 2010, Proceedings of the Royal Society B: Biological Sciences.

[26]  Attila Szolnoki,et al.  Phase diagrams for three-strategy evolutionary prisoner's dilemma games on regular graphs. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[27]  S. Kurokawa,et al.  Emergence of cooperation in public goods games , 2009, Proceedings of the Royal Society B: Biological Sciences.

[28]  F. C. Santos,et al.  Evolutionary dynamics of collective action in N-person stag hunt dilemmas , 2009, Proceedings of the Royal Society B: Biological Sciences.

[29]  Wolfgang J. Luhan,et al.  Cedex Discussion Paper Series , 2022 .

[30]  David G. Rand,et al.  Winners don’t punish , 2008, Nature.

[31]  Karl Sigmund,et al.  Punish or perish? Retaliation and collaboration among humans. , 2007, Trends in ecology & evolution.

[32]  Drew Fudenberg,et al.  Imitation Processes with Small Mutations , 2004, J. Econ. Theory.

[33]  M. Nowak,et al.  Stochastic dynamics of invasion and fixation. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[34]  C. Hauert,et al.  Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game , 2005 .

[35]  D. Fudenberg,et al.  Emergence of cooperation and evolutionary stability in finite populations , 2004, Nature.

[36]  Marcus W Feldman,et al.  What is altruism? , 2004, Trends in ecology & evolution.

[37]  Daniel B. Neill,et al.  Optimality under noise: higher memory strategies for the alternating prisoner's dilemma. , 2001, Journal of theoretical biology.

[38]  D P Kraines,et al.  Natural selection of memory-one strategies for the iterated prisoner's dilemma. , 2000, Journal of theoretical biology.

[39]  M. Milinski,et al.  Working memory constrains human cooperation in the Prisoner's Dilemma. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[40]  C. Hauert,et al.  Effects of increasing the number of players and memory size in the iterated Prisoner's Dilemma: a numerical approach , 1997, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[41]  Kristian Lindgren,et al.  Evolutionary dynamics in game-theoretic models , 1996 .

[42]  David P. Kraines,et al.  Learning to cooperate with Pavlov an adaptive strategy for the iterated Prisoner's Dilemma with noise , 1993 .

[43]  M. Nowak,et al.  A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's Dilemma game , 1993, Nature.

[44]  M. Nowak,et al.  Tit for tat in heterogeneous populations , 1992, Nature.

[45]  D. Fudenberg,et al.  Evolution and Cooperation in Noisy Repeated Games , 1990 .

[46]  Roger Ware,et al.  Evolutionary stability in the repeated prisoner's dilemma , 1989 .

[47]  R. Boyd Mistakes allow evolutionary stability in the repeated prisoner's dilemma game. , 1989, Journal of theoretical biology.

[48]  W. Arthur,et al.  The Economy as an Evolving Complex System II , 1988 .

[49]  R. Boyd,et al.  No pure strategy is evolutionarily stable in the repeated Prisoner's Dilemma game , 1987, Nature.

[50]  P. Molander The Optimal Level of Generosity in a Selfish, Uncertain Environment , 1985 .

[51]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[52]  J. M. Smith The theory of games and the evolution of animal conflicts. , 1974, Journal of theoretical biology.