An analysis of two-population coevolutionary computation

Coevolutionary computation (CoEC) is the subfield of evolutionary computation (EC) centered around the notion of interaction among simultaneously evolving entities. While it promises important problem-solving advantages, coevolution also brings many challenges. A practitioner trying to use a coevolutionary algorithm (CoEA) to solve a problem is generally interested in how the choices made in designing the algorithm affect its performance on that particular problem. In other words, they would like to have information about the dependency: problem properties + algorithm properties → performance. For traditional evolutionary algorithms (EAs), our understanding of this dependency has reached reasonably satisfactory levels. For CoEAs it has not, as they have proven notoriously more complex and less intuitive. The main contribution of my dissertation is advancing the understanding of this dependency for traditional two-population coevolutionary algorithms. The way I achieve this is through extensive analysis that connects algorithm, problem and performance through one key aspect: dynamics. While the importance of understanding the dynamics of coevolutionary systems has been pointed out by previous research, this dissertation is the first study that "glues" all four pieces together. Additionally, an important feature of the analysis is that it spans subareas of CoEC that were previously studied independently (compositional cooperative and test-based competitive). It bridges them by identifying a problem property and introducing tools for analyzing this property that are applicable across subareas, thus providing a more holistic perspective of the field of CoEC. The analysis is performed both for previously unstudied aspects of CoEAs and for ones that have been investigated using other techniques. The power of the new analysis approach is particularly visible in the latter case, where it is shown to explain prior "mysterious" results.

[1]  Larry Bull,et al.  Evolutionary computing in multi-agent environments: Partners , 1997 .

[2]  J. Pollack,et al.  Focusing versus Intransitivity Geometrical Aspects of Co-evolution , 2003 .

[3]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[4]  Hugues Juillé,et al.  Incremental Co-Evolution of Organisms: A New Approach for Optimization and Discovery of Strategies , 1995, ECAL.

[5]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[6]  Edwin D. de Jong,et al.  Intransitivity in Coevolution , 2004, PPSN.

[7]  R. Paul Wiegand,et al.  Spatial Embedding and Loss of Gradient in Cooperative Coevolutionary Algorithms , 2004, PPSN.

[8]  Lothar M. Schmitt Coevolutionary Convergence to Global Optima , 2003, GECCO.

[9]  Edwin D. de Jong,et al.  The Incremental Pareto-Coevolution Archive , 2004, GECCO.

[10]  Jayshree Sarma,et al.  An analysis of decentralized and spatially distributed genetic algorithms , 1998 .

[11]  Edwin D. de Jong,et al.  The parallel Nash Memory for asymmetric games , 2006, GECCO.

[12]  R. Paul Wiegand,et al.  An empirical analysis of collaboration methods in cooperative coevolutionary algorithms , 2001 .

[13]  J. Pollack,et al.  Coevolutionary dynamics in a minimal substrate , 2001 .

[14]  Sean Luke,et al.  A Comparison Of Two Competitive Fitness Functions , 2002, GECCO.

[15]  Kenneth A. De Jong,et al.  Understanding cooperative co-evolutionary dynamics via simple fitness landscapes , 2005, GECCO '05.

[16]  John H. Miller,et al.  The coevolution of automata in the repeated Prisoner's Dilemma , 1996 .

[17]  Hod Lipson,et al.  Nonlinear system identification using coevolution of models and tests , 2005, IEEE Transactions on Evolutionary Computation.

[18]  Paulien Hogeweg,et al.  Information integration and red queen dynamics in coevolutionary optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[19]  Kenneth A. De Jong,et al.  The effects of interaction frequency on the optimization performance of cooperative coevolution , 2006, GECCO.

[20]  Jordan B. Pollack,et al.  A Population-Differential Method of Monitoring Success and Failure in Coevolution , 2004, GECCO.

[21]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[22]  Thomas Jansen,et al.  Sequential versus parallel cooperative coevolutionary (1+1) EAs , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[23]  Jordan B. Pollack,et al.  On identifying global optima in cooperative coevolution , 2005, GECCO '05.

[24]  Kazuhiro Ohkura,et al.  Modeling Coevolutionary Genetic Algorithms on Two-Bit Landscapes: Random Partnering , 2004, GECCO.

[25]  Jordan B. Pollack,et al.  Effects of Finite Populations on Evolutionary Stable Strategies , 2000, GECCO.

[26]  Jordan B. Pollack,et al.  Pareto Optimality in Coevolutionary Learning , 2001, ECAL.

[27]  Kenneth A. De Jong,et al.  The dynamics of the best individuals in co-evolution , 2006, Natural Computing.

[28]  Sevan G. Ficici A game-theoretic investigation of selection methods in two-population coevolution , 2006, GECCO '06.

[29]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[30]  Lothar M. Schmitt,et al.  Theory of Coevolutionary Genetic Algorithms , 2003, ISPA.

[31]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[32]  Thomas Jansen,et al.  Exploring the Explorative Advantage of the Cooperative Coevolutionary (1+1) EA , 2003, GECCO.

[33]  Richard K. Belew,et al.  Coevolutionary search among adversaries , 1997 .

[34]  Gary B. Parker,et al.  Comparison of sampling sizes for the co-evolution of cooperative agents , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[35]  William M. Spears,et al.  Simple Subpopulation Schemes , 1998 .

[36]  Jeffrey K. Bassett,et al.  An Analysis of Cooperative Coevolutionary Algorithms A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University , 2003 .

[37]  J. Pollack,et al.  Coevolving the "Ideal" Trainer: Application to the Discovery of Cellular Automata Rules , 1998 .

[38]  S. Luke,et al.  When Coevolutionary Algorithms Exhibit Evolutionary Dyna mics , 2002 .

[39]  Edwin D. de Jong,et al.  The MaxSolve algorithm for coevolution , 2005, GECCO '05.

[40]  Beat Kleiner,et al.  Graphical Methods for Data Analysis , 1983 .

[41]  J. Yorke,et al.  Chaos: An Introduction to Dynamical Systems , 1997 .

[42]  Kenneth A. De Jong,et al.  The Effects of Representational Bias on Collaboration Methods in Cooperative Coevolution , 2002, PPSN.

[43]  Jordan B. Pollack,et al.  A Game-Theoretic Memory Mechanism for Coevolution , 2003, GECCO.

[44]  Peter A. J. Hilbers,et al.  Predicting Genetic Drift in 2×2 Games , 2004, GECCO.

[45]  Larry Bull,et al.  Coevolutionary species adaptation genetic algorithms: growth and mutation on coupled fitness landscapes , 2005, 2005 IEEE Congress on Evolutionary Computation.

[46]  F. Alajaji,et al.  c ○ Copyright by , 1998 .

[47]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[48]  Mats G. Nordahl,et al.  Coevolving pursuit-evasion strategies in open and confined regions , 1998 .

[49]  Seth Bullock,et al.  Combating Coevolutionary Disengagement by Reducing Parasite Virulence , 2004, Evolutionary Computation.

[50]  Carl T. Bergstrom,et al.  The Red King effect: When the slowest runner wins the coevolutionary race , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[51]  Richard K. Belew,et al.  Methods for Competitive Co-Evolution: Finding Opponents Worth Beating , 1995, ICGA.

[52]  J. Pollack,et al.  A game-theoretic investigation of selection methods used in evolutionary algorithms , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[53]  Edwin D. de Jong,et al.  Ideal Evaluation from Coevolution , 2004, Evolutionary Computation.

[54]  Kenneth A. De Jong,et al.  A Dynamical Systems Analysis of Collaboration Methods in Cooperative Co-evolution , 2005, AAAI Fall Symposium: Coevolutionary and Coadaptive Systems.

[55]  Edwin D. de Jong,et al.  Towards a bounded Pareto-coevolution archive , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[56]  Sean Luke,et al.  Lenience towards Teammates Helps in Cooperative Multiagent Learning , 2005 .

[57]  R. Paul Wiegand,et al.  Robustness in cooperative coevolution , 2006, GECCO '06.

[58]  Dave Cliff,et al.  Tracking the Red Queen: Measurements of Adaptive Progress in Co-Evolutionary Simulations , 1995, ECAL.

[59]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[60]  Edwin D. de Jong,et al.  DECA: dimension extracting coevolutionary algorithm , 2006, GECCO '06.

[61]  Josef Hofbauer,et al.  Evolutionary Games and Population Dynamics , 1998 .

[62]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[63]  R. Paul Wiegand,et al.  Improving Coevolutionary Search for Optimal Multiagent Behaviors , 2003, IJCAI.

[64]  J. Thompson,et al.  The Coevolutionary Process , 1994 .

[65]  Peter J. Angeline,et al.  Competitive Environments Evolve Better Solutions for Complex Tasks , 1993, ICGA.

[66]  Gary B. Parker,et al.  Punctuated anytime learning for evolving multi-agent capture strategies , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[67]  Sean Luke,et al.  Selecting informative actions improves cooperative multiagent learning , 2006, AAMAS '06.

[68]  Seth Bullock,et al.  Unpicking Tartan CIAO Plots: Understanding Irregular Coevolutionary Cycling , 2004, Adapt. Behav..

[69]  Melanie Mitchell,et al.  Investigating the success of spatial coevolution , 2005, GECCO '05.

[70]  Mitchell A. Potter,et al.  The design and analysis of a computational model of cooperative coevolution , 1997 .

[71]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[72]  Kenneth A. De Jong,et al.  Sequential versus Parallel Cooperative Coevolutionary Algorithms for Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[73]  Larry Bull On coevolutionary genetic algorithms , 2001, Soft Comput..

[74]  R. Paul Wiegand,et al.  A Sensitivity Analysis of a Cooperative Coevolutionary Algorithm Biased for Optimization , 2004, GECCO.

[75]  Y. Hochberg A sharper Bonferroni procedure for multiple tests of significance , 1988 .

[76]  J. Pollack,et al.  Challenges in coevolutionary learning: arms-race dynamics, open-endedness, and medicocre stable states , 1998 .

[77]  Stefano Nolfi,et al.  God Save the Red Queen! Competition in Co-Evolutionary Robotics , 1997 .

[78]  J. Pollack,et al.  Coevolving High-Level Representations , 1993 .

[79]  R. Axelrod Evolving New Strategies The Evolution of Strategies in the Iterated Prisoner ' s Dilemma , 1997 .

[80]  Björn Olsson,et al.  Co-evolutionary search in asymmetric spaces , 2001, Inf. Sci..

[81]  Pablo Funes,et al.  Intransitivity revisited coevolutionary dynamics of numbers games , 2005, GECCO '05.

[82]  Melanie Mitchell,et al.  A Comparison of Evolutionary and Coevolutionary Search , 2002, Int. J. Comput. Intell. Appl..

[83]  R. Paul Wiegand,et al.  A Visual Demonstration of Convergence Properties of Cooperative Coevolution , 2004, PPSN.

[84]  Kenneth A. De Jong,et al.  Relationships between internal and external metrics in co-evolution , 2005, 2005 IEEE Congress on Evolutionary Computation.

[85]  John R. Koza,et al.  Genetic evolution and co-evolution of computer programs , 1991 .

[86]  Bert Thompson,et al.  BIOLOGICAL EVOLUTION , 2004 .

[87]  Pattie Maes,et al.  Co-evolution of Pursuit and Evasion II: Simulation Methods and Results , 1996 .

[88]  Kenneth A. De Jong,et al.  Understanding EA Dynamics via Population Fitness Distributions , 2003, GECCO.

[89]  Seth Bullock,et al.  Learning lessons from the common cold: How reducing parasite virulence improves coevolutionary optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[90]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[91]  Peter J. Angeline,et al.  Adaptive and Self-adaptive Evolutionary Computations , 1995 .

[92]  Matt Ridley The Red Queen , 2000 .

[93]  Sean Luke,et al.  Time-dependent Collaboration Schemes for Cooperative Coevolutionary Algorithms , 2005, AAAI Fall Symposium: Coevolutionary and Coadaptive Systems.

[94]  Jordan B. Pollack,et al.  A game-theoretic and dynamical-systems analysis of selection methods in coevolution , 2005, IEEE Transactions on Evolutionary Computation.

[95]  Larry Bull,et al.  Coevolutionary Species Adaptation Genetic Algorithms: A Continuing SAGA on Coupled Fitness Landscapes , 2005, ECAL.

[96]  Jordan B. Pollack,et al.  Measuring Progress in Coevolutionary Competition , 2000 .

[97]  Seth Bullock,et al.  Caring versus Sharing: How to Maintain Engagement and Diversity in Coevolving Populations , 2003, ECAL.

[98]  Karl Sims,et al.  Evolving 3d morphology and behavior by competition , 1994 .

[99]  Craig W. Reynolds Competition, Coevolution and the Game of Tag , 1994 .

[100]  Sean Luke,et al.  Archive-based cooperative coevolutionary algorithms , 2006, GECCO '06.

[101]  J. Tukey,et al.  Variations of Box Plots , 1978 .

[102]  Kenneth O. Stanley and Joseph Reisinger and Risto Miikkulainen,et al.  The Dominance Tournament Method of Monitoring Progress in Coevolution , 2002 .

[103]  Jan Paredis,et al.  Coevolving Cellular Automata: Be Aware of the Red Queen! , 1997, ICGA.

[104]  Sean Luke,et al.  The analysis and design of concurrent learning algorithms for cooperative multiagent systems , 2007 .

[105]  Larry Bull,et al.  Evolutionary Computing in Multi-agent Environments: Operators , 1998, Evolutionary Programming.