Ideal Evaluation from Coevolution

In many problems of interest, performance can be evaluated using tests, such as examples in concept learning, test points in function approximation, and opponents in game-playing. Evaluation on all tests is often infeasible. Identification of an accurate evaluation or fitness function is a difficult problem in itself, and approximations are likely to introduce human biases into the search process. Coevolution evolves the set of tests used for evaluation, but has so far often led to inaccurate evaluation. We show that for any set of learners, a Complete Evaluation Set can be determined that provides ideal evaluation as specified by Evolutionary Multi-Objective Optimization. This provides a principled approach to evaluation in coevolution, and thereby brings automatic ideal evaluation within reach. The Complete Evaluation Set is of manageable size, and progress towards it can be accurately measured. Based on this observation, an algorithm named DELPHI is developed. The algorithm is tested on problems likely to permit progress on only a subset of the underlying objectives. Where all comparison methods result in overspecialization, the proposed method and a variant achieve sustained progress in all underlying objectives. These findings demonstrate that ideal evaluation may be approximated by practical algorithms, and that accurate evaluation for test-based problems is possible even when the underlying objectives of a problem are unknown.

[1]  J. Neumann Zur Theorie der Gesellschaftsspiele , 1928 .

[2]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[3]  Nils Aall Barricelli,et al.  Numerical testing of evolution theories , 1963 .

[4]  D. E. Matthews Evolution and the Theory of Games , 1977 .

[5]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[6]  John R. Koza,et al.  Evolution and co-evolution of computer programs to control independently-acting agents , 1991 .

[7]  John R. Koza,et al.  Genetic evolution and co-evolution of computer programs , 1991 .

[8]  Phil Husbands,et al.  Simulated Co-Evolution as the Mechanism for Emergent Planning and Scheduling , 1991, ICGA.

[9]  Jean-Arcady Meyer,et al.  Evolution and Co Evolution of Computer Programs to Control Independently Acting Agents , 1991 .

[10]  S. Kauffman,et al.  Coevolution to the edge of chaos: coupled fitness landscapes, poised states, and coevolutionary avalanches. , 1991, Journal of theoretical biology.

[11]  G. M. Werner Evolution of Communication in Artificial Organisms, Artifial Life II , 1991 .

[12]  Kristian Lindgren,et al.  Evolutionary phenomena in simple dynamics , 1992 .

[13]  Peter J. Angeline,et al.  Competitive Environments Evolve Better Solutions for Complex Tasks , 1993, ICGA.

[14]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[15]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[16]  J. Pollack,et al.  Coevolving High-Level Representations , 1993 .

[17]  Dave Cliff,et al.  Protean behavior in dynamic games: arguments for the co-evolution of pursuit-evasion tactics , 1994 .

[18]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[19]  Susan L. Epstein Toward an Ideal Trainer , 1994 .

[20]  Phil Husbands,et al.  Distributed Coevolutionary Genetic Algorithms for Multi-Criteria and Multi-Constraint Optimisation , 1994, Evolutionary Computing, AISB Workshop.

[21]  Karl Sims,et al.  Evolving 3d morphology and behavior by competition , 1994 .

[22]  Craig W. Reynolds Competition, Coevolution and the Game of Tag , 1994 .

[23]  Dave Cliff,et al.  Tracking the Red Queen: Measurements of Adaptive Progress in Co-Evolutionary Simulations , 1995, ECAL.

[24]  Peter J. Fleming,et al.  An Overview of Evolutionary Algorithms in Multiobjective Optimization , 1995, Evolutionary Computation.

[25]  Jan Paredis,et al.  Coevolutionary Computation , 1995, Artificial Life.

[26]  Jordan B. Pollack,et al.  Co-Evolving Intertwined Spirals , 1996, Evolutionary Programming.

[27]  Samir W. Mahfoud Niching methods for genetic algorithms , 1996 .

[28]  Justinian P. Rosca,et al.  Discovery of subroutines in genetic programming , 1996 .

[29]  Pattie Maes,et al.  Co-evolution of Pursuit and Evasion II: Simulation Methods and Results , 1996 .

[30]  John H. Miller,et al.  The coevolution of automata in the repeated Prisoner's Dilemma , 1996 .

[31]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[32]  Richard K. Belew,et al.  Coevolutionary search among adversaries , 1997 .

[33]  Jan Paredis,et al.  Coevolving Cellular Automata: Be Aware of the Red Queen! , 1997, ICGA.

[34]  Paulien Hogeweg,et al.  Evolutionary Consequences of Coevolving Targets , 1997, Evolutionary Computation.

[35]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[36]  Jean-Arcady Meyer,et al.  Coevolving Communicative Behavior in a Linear Pursuer-Evader Game , 1998 .

[37]  Jordan B. Pollack,et al.  Coevolutionary Learning: A Case Study , 1998, ICML.

[38]  Stefano Nolfi,et al.  Co-evolving predator and prey robots , 1998, Artificial Life.

[39]  Stefano Nolfi,et al.  Competitive co-evolutionary robotics: from theory to practice , 1998 .

[40]  Jordan B. Pollack,et al.  Coevolving communicative behavior in a linear pursuer-evadergame , 1998 .

[41]  Jürgen Schmidhuber,et al.  Artificial curiosity based on discovering novel algorithmic predictability through coevolution , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[42]  Jordan B. Pollack,et al.  Methods for statistical inference: extending the evolutionary computation paradigm , 1999 .

[43]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[44]  David Corne,et al.  The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[45]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[46]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[47]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[48]  Carlos A. Coello Coello,et al.  An updated survey of GA-based multiobjective optimization techniques , 2000, CSUR.

[49]  Jordan B. Pollack,et al.  Symbiotic Combination as an Alternative to Sexual Recombination in Genetic Algorithms , 2000, PPSN.

[50]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[51]  J. Pollack,et al.  A game-theoretic investigation of selection methods used in evolutionary algorithms , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[52]  James P. Crutchfield,et al.  Resource sharing and coevolution in evolving cellular automata , 1999, IEEE Trans. Evol. Comput..

[53]  Martin J. Oates,et al.  The Pareto Envelope-Based Selection Algorithm for Multi-objective Optimisation , 2000, PPSN.

[54]  Larry Bull,et al.  Coevolving functions in genetic programming , 2001, J. Syst. Archit..

[55]  Jordan B. Pollack,et al.  A Game-Theoretic Approach to the Simple Coevolutionary Algorithm , 2000, PPSN.

[56]  Paulien Hogeweg,et al.  Information integration and red queen dynamics in coevolutionary optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[57]  J. Pollack,et al.  Coevolutionary dynamics in a minimal substrate , 2001 .

[58]  J. Pollack,et al.  Evolution of complexity in real-world domains , 2001 .

[59]  Richard A. Watson,et al.  Reducing Local Optima in Single-Objective Problems by Multi-objectivization , 2001, EMO.

[60]  Björn Olsson,et al.  Co-evolutionary search in asymmetric spaces , 2001, Inf. Sci..

[61]  SharingSevan G. Fi,et al.  Game Theory and the Simple Coevolutionary Algorithm : SomePreliminary Results on Fitness , 2001 .

[62]  Jordan B. Pollack,et al.  Pareto Optimality in Coevolutionary Learning , 2001, ECAL.

[63]  R. Watson,et al.  Pareto coevolution: using performance against coevolved opponents in a game as dimensions for Pareto selection , 2001 .

[64]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[65]  Alex Lubberts and Risto Miikkulainen Co-Evolving a Go-Playing Neural network , 2001 .

[66]  Larry Bull On coevolutionary genetic algorithms , 2001, Soft Comput..

[67]  Jordan B. Pollack,et al.  A Mathematical Framework for the Study of Coevolution , 2002, FOGA.

[68]  Marco Laumanns,et al.  Combining Convergence and Diversity in Evolutionary Multiobjective Optimization , 2002, Evolutionary Computation.

[69]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[70]  Risto Miikkulainen,et al.  Continual Coevolution Through Complexification , 2002, GECCO.

[71]  J. Pollack,et al.  A computational model of symbiotic composition in evolutionary transitions. , 2003, Bio Systems.

[72]  Jordan B. Pollack,et al.  Focusing versus Intransitivity , 2003, GECCO.

[73]  Focusing versus Intransitivity Geometrical Aspects of Co-evolution , 2003 .

[74]  Henrik Hautop Lund,et al.  Co-evolving Complex Robot Behavior , 2003, ICES.

[75]  Edwin D. de Jong,et al.  Representation Development from Pareto-Coevolution , 2003, GECCO.

[76]  Jordan B. Pollack,et al.  Co-Evolution in the Successful Learning of Backgammon Strategy , 1998, Machine Learning.

[77]  U. Aickelin,et al.  Parallel Problem Solving from Nature - PPSN VIII , 2004, Lecture Notes in Computer Science.

[78]  David H. Wolpert,et al.  Coevolutionary free lunches , 2005, IEEE Transactions on Evolutionary Computation.

[79]  Hod Lipson,et al.  Nonlinear system identification using coevolution of models and tests , 2005, IEEE Transactions on Evolutionary Computation.

[80]  R. Paul Wiegand,et al.  Biasing Coevolutionary Search for Optimal Multiagent Behaviors , 2006, IEEE Transactions on Evolutionary Computation.

[81]  Raymond Y. K. Lau,et al.  An evolutionary learning approach for adaptive negotiation agents , 2006, Int. J. Intell. Syst..