Selection methods regulate evolution of cooperation in digital evolution

A key, yet often neglected, component of digital evolution and evolutionary models is the ‘selection method’ which assigns fitness (number of offspring) to individuals based on their performance scores (efficiency in performing tasks). Here, we study with formal analysis and numerical experiments the evolution of cooperation under the five most common selection methods (proportionate, rank, truncation-proportionate, truncation-uniform and tournament). We consider related individuals engaging in a Prisoner's Dilemma game where individuals can either cooperate or defect. A cooperator pays a cost, whereas its partner receives a benefit, which affect their performance scores. These performance scores are translated into fitness by one of the five selection methods. We show that cooperation is positively associated with the relatedness between individuals under all selection methods. By contrast, the change in the performance benefit of cooperation affects the populations’ average level of cooperation only under the proportionate methods. We also demonstrate that the truncation and tournament methods may introduce negative frequency-dependence and lead to the evolution of polymorphic populations. Using the example of the evolution of cooperation, we show that the choice of selection method, though it is often marginalized, can considerably affect the evolutionary dynamics.

[1]  N. Pierce Origin of Species , 1914, Nature.

[2]  P. Moran,et al.  The statistical processes of evolutionary theory. , 1963 .

[3]  A. W. F. Edwards,et al.  The statistical processes of evolutionary theory , 1963 .

[4]  W. Hamilton The genetical evolution of social behaviour. II. , 1964, Journal of theoretical biology.

[5]  W. Hamilton The genetical evolution of social behaviour. I. , 1964, Journal of theoretical biology.

[6]  M. Kimura,et al.  An introduction to population genetics theory , 1971 .

[7]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[8]  C. Brooke Worth,et al.  The Insect Societies , 1973 .

[9]  J. Gillespie Natural selection for within-generation variance in offspring number II. Discrite haploid models. , 1975, Genetics.

[10]  J. Gillespie Natural Selection for Variances in Offspring Numbers: A New Evolutionary Principle , 1977, The American Naturalist.

[11]  R. Milkman Selection differentials and selection coefficients. , 1978, Genetics.

[12]  J. Crow,et al.  Efficiency of truncation selection. , 1979, Proceedings of the National Academy of Sciences of the United States of America.

[13]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[14]  L. Cavalli-Sforza,et al.  Assortment of encounters and evolution of cooperativeness. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[15]  D. Queller Kinship, reciprocity and synergism in the evolution of social behaviour , 1985, Nature.

[16]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[17]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[18]  John Maynard Smith,et al.  Byte-sized evolution , 1992, Nature.

[19]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[20]  David B. Fogel,et al.  Evolving Behaviors in the Iterated Prisoner's Dilemma , 1993, Evolutionary Computation.

[21]  B A Huberman,et al.  Evolutionary games and computer simulations. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[22]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[23]  Thomas Bäck,et al.  Selective Pressure in Evolutionary Algorithms: A Characterization of Selection Mechanisms , 1994, International Conference on Evolutionary Computation.

[24]  Claude Lattaud,et al.  The Artificial Evolution of Cooperation , 1995, Artificial Evolution.

[25]  Lothar Thiele,et al.  A Mathematical Analysis of Tournament Selection , 1995, ICGA.

[26]  L. Altenberg,et al.  PERSPECTIVE: COMPLEX ADAPTATIONS AND THE EVOLUTION OF EVOLVABILITY , 1996, Evolution; international journal of organic evolution.

[27]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[28]  Arvin Agah,et al.  Phylogenetic and Ontogenetic Learning in a Colony of Interacting Robots , 1997, Auton. Robots.

[29]  Angelo Cangelosi,et al.  The Emergence of a 'Language' in an Evolving Population of Neural Networks , 1998, Connect. Sci..

[30]  Stefano Nolfi,et al.  Co-evolving predator and prey robots , 1998, Artificial Life.

[31]  O. Leimar,et al.  Evolution of cooperation through indirect reciprocity , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[32]  R. Riolo,et al.  Evolution of cooperation without reciprocity , 2001, Nature.

[33]  J. Foster Computational genetics: Evolutionary computation , 2001, Nature Reviews Genetics.

[34]  Graham Kendall,et al.  Evolving Collective Behavior in an Artificial Ecology , 2001, Artificial Life.

[35]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[36]  Angelo Cangelosi,et al.  Evolution of communication and language using signals, symbols, and words , 2001, IEEE Trans. Evol. Comput..

[37]  Inman Harvey,et al.  An Evolutionary Ecological Approach to the Study of Learning Behavior Using a Robot-Based Model , 2002, Adapt. Behav..

[38]  Isaac Meilijson,et al.  Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors , 2002, Adapt. Behav..

[39]  Stefano Nolfi,et al.  The emergence of communication in evolutionary robots , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[40]  Stefano Nolfi,et al.  Evolving Mobile Robots Able to Display Collective Behaviors , 2003, Artificial Life.

[41]  Lincoln Smith,et al.  Evolving controllers for a homogeneous system of physical robots: structured cooperation with minimal sensors , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[42]  Sean H. Rice,et al.  Evolutionary Theory: Mathematical and Conceptual Foundations , 2004 .

[43]  Michael Doebeli,et al.  Spatial structure often inhibits the evolution of cooperation in the snowdrift game , 2004, Nature.

[44]  H. Gintis,et al.  The evolution of strong reciprocity: cooperation in heterogeneous populations. , 2004, Theoretical population biology.

[45]  Risto Miikkulainen,et al.  Online Interactive Neuro-evolution , 2000, Neural Processing Letters.

[46]  A. Griffin,et al.  Cooperation and competition in pathogenic bacteria , 2004, Nature.

[47]  D. Greig,et al.  The Prisoner's Dilemma and polymorphism in yeast SUC genes , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[48]  Martin A. Nowak,et al.  Evolutionary dynamics on graphs , 2005, Nature.

[49]  Jordan B. Pollack,et al.  A game-theoretic and dynamical-systems analysis of selection methods in coevolution , 2005, IEEE Transactions on Evolutionary Computation.

[50]  D. DeAngelis,et al.  Individual-based modeling of ecological and evolutionary processes , 2005 .

[51]  P. Taylor Altruism in viscous populations — an inclusive fitness model , 1992, Evolutionary Ecology.

[52]  G. Robinson,et al.  Sociogenomics: social life in molecular terms , 2005, Nature Reviews Genetics.

[53]  C. Hauert,et al.  Models of cooperation based on the Prisoner's Dilemma and the Snowdrift game , 2005 .

[54]  Günter P. Wagner,et al.  Complex Adaptations and the Evolution of Evolvability , 2005 .

[55]  M. Nowak Evolutionary Dynamics: Exploring the Equations of Life , 2006 .

[56]  C. Adami Digital genetics: unravelling the genetic basis of evolution , 2006, Nature Reviews Genetics.

[57]  A. Griffin,et al.  Social evolution theory for microorganisms , 2006, Nature Reviews Microbiology.

[58]  D. Floreano,et al.  Division of labour and colony efficiency in social insects: effects of interactions between genetic architecture, colony kin structure and rate of perturbations , 2006, Proceedings of the Royal Society B: Biological Sciences.

[59]  L. Keller,et al.  The evolution of cooperation and altruism – a general framework and a classification of models , 2006, Journal of evolutionary biology.

[60]  D. Floreano,et al.  Evolutionary Conditions for the Emergence of Communication in Robots , 2007, Current Biology.

[61]  Uri Alon,et al.  Varying environments can speed up evolution , 2007, Proceedings of the National Academy of Sciences.

[62]  L. Lehmann,et al.  Natural Selection on Fecundity Variance in Subdivided Populations: Kin Selection Meets Bet Hedging , 2007, Genetics.

[63]  Colm O'Riordan,et al.  Evolving team behaviours in environments of varying difficulty , 2007, Artificial Intelligence Review.

[64]  S. Rice A stochastic version of the Price equation reveals the interplay of deterministic and stochastic processes in evolution , 2008, BMC Evolutionary Biology.

[65]  L. Hurst Genetics and the understanding of selection , 2009, Nature Reviews Genetics.

[66]  B. Kerr,et al.  Selection in Ephemeral Networks , 2009, The American Naturalist.

[67]  D. Floreano,et al.  The evolution of information suppression in communicating robots with conflicting interests , 2009, Proceedings of the National Academy of Sciences.

[68]  Dario Floreano,et al.  Genetic Team Composition and Level of Selection in the Evolution of Cooperation , 2009, IEEE Transactions on Evolutionary Computation.

[69]  D. Floreano,et al.  Task-dependent influence of genetic architecture and mating frequency on division of labour in social insect societies , 2010, Behavioral Ecology and Sociobiology.

[70]  D. Floreano,et al.  Evolution of Adaptive Behaviour in Robots by Means of Darwinian Selection , 2010, PLoS biology.

[71]  Arend Hintze,et al.  Critical Dynamics in the Evolution of Stochastic Strategies for the Iterated Prisoner's Dilemma , 2010, PLoS Comput. Biol..

[72]  M. Genovart,et al.  The Young, the Weak and the Sick: Evidence of Natural Selection by Predation , 2010, PloS one.

[73]  J. Slate,et al.  Natural and Sexual Selection in a Wild Insect Population , 2010, Science.

[74]  Alex A. Freitas,et al.  Evolutionary Computation , 2002 .

[75]  G. Wagner THE MEASUREMENT THEORY OF FITNESS , 2010, Evolution; international journal of organic evolution.

[76]  Heather Goldsby,et al.  Selective pressures for accurate altruism targeting: evidence from digital evolution for difficult-to-test aspects of inclusive fitness theory , 2011, Proceedings of the Royal Society B: Biological Sciences.

[77]  Richard A. Watson,et al.  THE CONCURRENT EVOLUTION OF COOPERATION AND THE POPULATION STRUCTURES THAT SUPPORT IT , 2011, Evolution; international journal of organic evolution.

[78]  D. Floreano,et al.  A Quantitative Test of Hamilton's Rule for the Evolution of Altruism , 2011, PLoS biology.

[79]  R. Michod,et al.  Inclusive fitness in evolution , 2011, Nature.

[80]  T. Schoener The Newest Synthesis: Understanding the Interplay of Evolutionary and Ecological Dynamics , 2011, Science.

[81]  F. Zhang,et al.  Eco-Evolutionary Feedback and the Invasion of Cooperation in Prisoner's Dilemma Games , 2011, PloS one.

[82]  Sara Mitri,et al.  Social evolution in multispecies biofilms , 2011, Proceedings of the National Academy of Sciences.

[83]  D. Floreano,et al.  Historical contingency affects signaling strategies and competitive abilities in evolving populations of simulated robots , 2012, Proceedings of the National Academy of Sciences.

[84]  C. Ofria,et al.  Task-switching costs promote the evolution of division of labor and shifts in individuality , 2012, Proceedings of the National Academy of Sciences.

[85]  Franz J. Weissing,et al.  University of Groningen Implications of behavioral architecture for the evolution of self-organized division of labor , 2019 .

[86]  Dario Floreano,et al.  Neural Networks as Mechanisms to Regulate Division of Labor , 2012, The American Naturalist.

[87]  Dario Floreano,et al.  Using robots to understand social behaviour , 2013, Biological reviews of the Cambridge Philosophical Society.