A game-theoretic and dynamical-systems analysis of selection methods in coevolution

We use evolutionary game theory (EGT) to investigate the dynamics and equilibria of selection methods in coevolutionary algorithms. The canonical selection method used in EGT is equivalent to the standard "fitness-proportional" selection method used in evolutionary algorithms. All attractors of the EGT dynamic are Nash equilibria; we focus on simple symmetric variable-sum games that have polymorphic Nash-equilibrium attractors. Against the dynamics of proportional selection, we contrast the behaviors of truncation selection, (/spl mu/,/spl lambda/),(/spl mu/+/spl lambda/), linear ranking, Boltzmann, and tournament selection. Except for Boltzmann selection, each of the methods we test unconditionally fail to achieve polymorphic Nash equilibrium. Instead, we find point attractors that lack game-theoretic justification, cyclic dynamics, or chaos. Boltzmann selection converges onto polymorphic Nash equilibrium only when selection pressure is sufficiently low; otherwise, we obtain attracting limit-cycles or chaos. Coevolutionary algorithms are often used to search for solutions (e.g., Nash equilibria) of games of strategy; our results show that many selection methods are inappropriate for finding polymorphic Nash solutions to variable-sum games. Another application of coevolution is to model other systems; our results emphasize the degree to which the model's behavior is sensitive to implementation details regarding selection-details that we might not otherwise believe to be critical.

[1]  Seth Bullock,et al.  Learning lessons from the common cold: How reducing parasite virulence improves coevolutionary optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  Craig W. Reynolds.  Competition, Coevolution and the Game of Tag , 1994 .

[3]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[4]  T. Yi,et al.  Evolutionarily stable strategy, stable state, periodic cycle and chaos in a simple discrete time two-phenotype model. , 1997, Journal of theoretical biology.

[5]  Nicholas C. Metropolis,et al.  On Finite Limit Sets for Transformations on the Unit Interval , 1973, J. Comb. Theory, Ser. A.

[6]  Martin A. Nowak,et al.  Game-dynamical aspects of the prisoner's dilemma , 1989 .

[7]  Robert W. Easton,et al.  Geometric methods for discrete dynamical systems , 1998 .

[8]  C. Cannings,et al.  The Generalized War of Attrition , 1997 .

[9]  Jordan B. Pollack,et al.  Effects of Finite Populations on Evolutionary Stable Strategies , 2000, GECCO.

[10]  R. Lewontin.  ‘The Selfish Gene’ , 1977, Nature.

[11]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[12]  Hartmut Jürgens,et al.  Chaos and Fractals: New Frontiers of Science , 1992 .

[13]  Bruce Tidor,et al.  An Analysis of Selection Procedures with Particular Attention Paid to Proportional and Boltzmann Selection , 1993, International Conference on Genetic Algorithms.

[14]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[15]  Paulien Hogeweg,et al.  Evolutionary Consequences of Coevolving Targets , 1997, Evolutionary Computation.

[16]  D.B. Fogel,et al.  A self-learning evolutionary chess program , 2004, Proceedings of the IEEE.

[17]  Peter J. B. Hancock,et al.  An Empirical Comparison of Selection Methods in Evolutionary Algorithms , 1994, Evolutionary Computing, AISB Workshop.

[18]  David B. Fogel,et al.  Evolutionary Stable Strategies Are Not Always Stable under Evolutionary Dynamics , 1995, Evolutionary Programming.

[19]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[20]  Lothar Thiele,et al.  A Comparison of Selection Schemes Used in Evolutionary Algorithms , 1996, Evolutionary Computation.

[21]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[22]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[23]  Stefano Nolfi,et al.  Co-evolving predator and prey robots , 1998, Artificial Life.

[24]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[25]  D. Fudenberg,et al.  Emergence of cooperation and evolutionary stability in finite populations , 2004, Nature.

[26]  James E. Baker,et al.  Adaptive Selection Methods for Genetic Algorithms , 1985, International Conference on Genetic Algorithms.

[27]  David B. Fogel,et al.  An Overview of Evolutionary Programming , 1999 .

[28]  Samir W. Mahfoud.  Boltzmann selection , 2018, Evolutionary Computation 1.

[29]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[30]  J. Pollack,et al.  Coevolving the "Ideal" Trainer: Application to the Discovery of Cellular Automata Rules , 1998 .

[31]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[32]  R. Riolo,et al.  Evolution of cooperation without reciprocity , 2001, Nature.

[33]  Thomas Bäck,et al.  Evolution Strategies: An Alternative Evolutionary Algorithm , 1995, Artificial Evolution.

[34]  Sadri Hassani,et al.  Nonlinear Dynamics and Chaos , 2000 .

[35]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[36]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[37]  Jordan B. Pollack,et al.  Co-Evolution in the Successful Learning of Backgammon Strategy , 1998, Machine Learning.

[38]  L. Samuelson.  Evolutionary Games and Equilibrium Selection , 1997 .

[39]  Brian Skyrms,et al.  Chaos and the Explanatory Significance of Equilibrium: Strange Attractors in Evolutionary Game Dynamics , 1992, PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association.

[40]  David B. Fogel.  Other selection methods , 1997 .

[41]  C. McDiarmid.  SIMULATED ANNEALING AND BOLTZMANN MACHINES A Stochastic Approach to Combinatorial Optimization and Neural Computing , 1991 .

[42]  E Cantu Paz.  ORDER STATISTICS AND SELECTION METHODS OF EVOLUTIONARY ALGORITHMS , 2002 .

[43]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[44]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[45]  David B. Fogel,et al.  Evolving neural networks to play checkers without relying on expert knowledge , 1999, IEEE Trans. Neural Networks.

[46]  A. Katok,et al.  Introduction to the Modern Theory of Dynamical Systems: INTRODUCTION , 1995 .

[47]  David B. Fogel,et al.  On the instability of evolutionary stable strategies in small populations , 1998 .

[48]  Peter J. Angeline,et al.  Competitive Environments Evolve Better Solutions for Complex Tasks , 1993, ICGA.

[49]  Erick Cantú-Paz,et al.  Order statistics and selection methods of evolutionary algorithms , 2002, Inf. Process. Lett..

[50]  Claude Lattaud,et al.  The Artificial Evolution of Cooperation , 1995, Artificial Evolution.

[51]  John J. Grefenstette,et al.  Rank-based selection , 2018, Evolutionary Computation 1.

[52]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[53]  D. Fogel,et al.  On the instability of evolutionary stable strategies. , 1997, Bio Systems.

[54]  D B Fogel,et al.  Do evolutionary processes minimize expected losses? , 2000, Journal of theoretical biology.

[55]  F. Alajaji,et al.  c ○ Copyright by , 1998 .

[56]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[57]  Dietmar Saupe,et al.  Chaos and fractals - new frontiers of science , 1992 .

[58]  J. Pollack,et al.  A game-theoretic investigation of selection methods used in evolutionary algorithms , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).