No Strategy Can Win in the Repeated Prisoner's Dilemma: Linking Game Theory and Computer Simulations

Computer simulations are regularly used for studying the evolution of strategies in repeated games. These simulations rarely pay attention to game theoretical results that can illuminate the data analysis or the questions being asked. Results from evolutionary game theory imply that for every Nash equilibrium, there are sequences of mutants that would destabilize them. If strategies are not limited to a finite set, populations move between a variety of Nash equilibria with different levels of cooperation. This instability is inescapable, regardless of how strategies are represented. We present algorithms that show that simulations do agree with the theory. This implies that cognition itself may only have limited impact on the cycling dynamics. We argue that the role of mutations or exploration is more important in determining levels of cooperation.

[1]  J Bendor,et al.  Types of evolutionary stability and the problem of cooperation. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[2]  F. C. Santos,et al.  Stochastic Dynamics through Hierarchically Embedded Markov Chains. , 2017, Physical review letters.

[3]  D. Fudenberg,et al.  Evolution and Cooperation in Noisy Repeated Games , 1990 .

[4]  J. Thoday.  Population Genetics , 1956, Nature.

[5]  D. Fudenberg,et al.  Evolutionary cycles of cooperation and defection. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Julián García,et al.  In and out of equilibrium I: Evolution of strategies in repeated games with discounting , 2007, J. Econ. Theory.

[7]  A. Gray,et al.  I. THE ORIGIN OF SPECIES BY MEANS OF NATURAL SELECTION , 1963 .

[8]  Kristian Lindgren,et al.  Evolutionary phenomena in simple dynamics , 1992 .

[9]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[10]  R. Axelrod.  Reviews book & software , 2022 .

[11]  Drew Fudenberg,et al.  The Folk Theorem in Repeated Games with Discounting or with Incomplete Information , 1986 .

[12]  Martin A Nowak,et al.  Evolution of extortion in Iterated Prisoner’s Dilemma games , 2012, Proceedings of the National Academy of Sciences.

[13]  John Nachbar,et al.  Non-computable strategies and discounted repeated games , 1996 .

[14]  Robert Axelrod.  Die Evolution der Kooperation , 1988 .

[15]  David J. Cooper,et al.  Supergames Played by Finite Automata with Finite Costs of Complexity in an Evolutionary Setting , 1996 .

[16]  Karl H. Schlag,et al.  Evolutionarily stable sets , 2000, Int. J. Game Theory.

[17]  A. Rubinstein,et al.  The Structure of Nash Equilibrium in Repeated Games with Finite Automata (Now published in Econometrica, 56 (1988), pp.1259-1282.) , 1986 .

[18]  Ross Cressman,et al.  Evolutionary Stability in the Finitely Repeated Prisoner 's Dilemma Game , 1996 .

[19]  M. Nowak,et al.  The evolution of stochastic strategies in the Prisoner's Dilemma , 1990 .

[20]  I. Gilboa.  The Complexity of Computing Best-Response Automata in Repeated Games , 1988 .

[21]  Andrew M. Colman,et al.  The complexity of cooperation: Agent-based models of competition and collaboration , 1998, Complex..

[22]  David G. Rand,et al.  Direct reciprocity in structured populations , 2012, Proceedings of the National Academy of Sciences.

[23]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[24]  Julián García,et al.  In and Out of Equilibrium II: Evolution in Repeated Games with Discounting and Complexity Costs , 2012, Games Econ. Behav..

[25]  R. Axelrod,et al.  Evolutionary Dynamics , 2004 .

[26]  M. Nowak.  Five Rules for the Evolution of Cooperation , 2006, Science.

[27]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization approaches to coevolve strategies for the iterated prisoner's dilemma , 2005, IEEE Transactions on Evolutionary Computation.

[28]  M. Nowak,et al.  A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner's Dilemma game , 1993, Nature.

[29]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[30]  B. Thomas.  On evolutionarily stable sets , 1985 .

[31]  Arne Traulsen,et al.  Partners or rivals? Strategies for the iterated prisoner's dilemma☆ , 2015, Games Econ. Behav..

[32]  C. Papadimitriou.  On platers with a bounded number of states , 1992 .

[33]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[34]  David B. Fogel,et al.  Evolving Behaviors in the Iterated Prisoner's Dilemma , 1993, Evolutionary Computation.

[35]  W. Press,et al.  Iterated Prisoner’s Dilemma contains strategies that dominate any evolutionary opponent , 2012, Proceedings of the National Academy of Sciences.

[36]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[37]  Academisch Proefschrift,et al.  The Moral Herd: Groups and the Evolution of Altruism and Cooperation , 2011 .

[38]  Elizabeth Pennisi,et al.  How Did Cooperative Behavior Evolve? , 2005, Science.

[39]  W. Ewens.  Mathematical Population Genetics : I. Theoretical Introduction , 2004 .

[40]  Francisco C. Santos,et al.  Intention recognition promotes the emergence of cooperation , 2011, Adapt. Behav..

[41]  L. Samuelson,et al.  Evolutionary stability in repeated games played by finite automata , 1992 .

[42]  Elchanan Ben-Porath.  The complexity of computing a best response automaton in repeated games with mixed strategies , 1990 .

[43]  Larry Samuelson,et al.  Evolutionary stability and lexicographic preferences , 2003, Games Econ. Behav..

[44]  Arne Traulsen,et al.  The Structure of Mutations and the Evolution of Cooperation , 2012, PloS one.

[45]  A. Rubinstein.  Finite automata play the repeated prisoner's dilemma , 1986 .

[46]  Oscar Volij,et al.  In Defense of DEFECT , 2002, Games Econ. Behav..

[47]  David B. Fogel,et al.  On the instability of evolutionary stable strategies in small populations , 1998 .

[48]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[49]  Xin Yao,et al.  The Iterated Prisoners' Dilemma - 20 Years On , 2007, Advances in Natural Computation.

[50]  Martin Jones,et al.  Reinforcement learning produces dominant strategies for the Iterated Prisoner’s Dilemma , 2017, PloS one.

[51]  D. M. V. Hesteren.  Evolutionary Game Theory , 2017 .

[52]  Alan Grafen,et al.  The hawk-dove game played between relatives , 1979, Animal Behaviour.

[53]  David B. Fogel,et al.  Evolutionary computation - toward a new philosophy of machine intelligence (3. ed.) , 1995 .

[54]  M. Nowak,et al.  Evolutionary game dynamics in a Wright-Fisher process , 2006, Journal of mathematical biology.

[55]  Matthijs van Veelen,et al.  Robustness against indirect invasions , 2012, Games Econ. Behav..

[56]  Xin Yao,et al.  On Evolving Robust Strategies for Iterated Prisoner's Dilemma , 1993, Evo Workshops.

[57]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[58]  P. Taylor,et al.  Evolutionarily Stable Strategies and Game Dynamics , 1978 .

[59]  Robert Axelrod.  Die Evolution der Kooperation: Aus dem Amerikanischen übersetzt und mit einem Nachwort von Werner Raub und Thomas Voss , 2009 .

[60]  Larry Samuelson,et al.  Evolution and Game Theory , 2002 .

[61]  R. Selten,et al.  Gaps in Harley's argument on evolutionarily stable learning rules and in the logic of “tit for tat” , 1984, Behavioral and Brain Sciences.

[62]  M. Nowak,et al.  Tit for tat in heterogeneous populations , 1992, Nature.

[63]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[64]  John H. Miller,et al.  The coevolution of automata in the repeated Prisoner's Dilemma , 1996 .

[65]  Chaitanya S. Gokhale,et al.  How small are small mutation rates? , 2011, Journal of Mathematical Biology.