Coevolutionary free lunches

Recent work on the foundational underpinnings of black-box optimization has begun to uncover a rich mathematical structure. In particular, it is now known that an inner product between the optimization algorithm and the distribution of optimization problems likely to be encountered fixes the distribution over likely performances in running that algorithm. One ramification of this is the "No Free Lunch" (NFL) theorems, which state that any two algorithms are equivalent when their performance is averaged across all possible problems. This highlights the need for exploiting problem-specific knowledge to achieve better than random performance. In this paper, we present a general framework covering most optimization scenarios. In addition to the optimization scenarios addressed in the NFL results, this framework covers multiarmed bandit problems and evolution of multiple coevolving players. As a particular instance of the latter, it covers "self-play" problems. In these problems, the set of players work together to produce a champion, who then engages one or more antagonists in a subsequent multiplayer game. In contrast to the traditional optimization case where the NFL results hold, we show that in self-play there are free lunches: in coevolution some algorithms have better performance than other algorithms, averaged across all possible problems. However, in the typical coevolutionary scenarios encountered in biology, where there is no champion, the NFL theorems still hold.

[1]  Nils Aall Barricelli,et al.  Numerical testing of evolution theories , 1963 .

[2]  J. Reed,et al.  Simulation of biological evolution and machine learning. I. Selection of self-reproducing numeric patterns by data processing machines, effects of hereditary control, mutation type and crossing. , 1967, Journal of theoretical biology.

[3]  George H. Burgin,et al.  COMPETITIVE GOAL-SEEKING THROUGH EVOLUTIONARY?PROGRAMMING. , 1969 .

[4]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[5]  D. E. Matthews.  Evolution and the Theory of Games , 1977 .

[6]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[7]  S. Hart,et al.  Handbook of Game Theory with Economic Applications , 1992 .

[8]  Susan L. Epstein.  Toward an Ideal Trainer , 1994 .

[9]  David H. Wolpert,et al.  The Relationship Between PAC, the Statistical Physics Framework, the Bayesian Framework, and the VC Framework , 1995 .

[10]  Patrick D. Surry,et al.  Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective , 1995, Computer Science Today.

[11]  David H. Wolpert,et al.  The Existence of A Priori Distinctions Between Learning Algorithms , 1996, Neural Computation.

[12]  David H. Wolpert,et al.  What makes an optimization problem hard? , 1995, Complex..

[13]  David H. Wolpert,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996, Neural Computation.

[14]  David B. Fogel,et al.  A note on representations and variation operators , 1997, IEEE Trans. Evol. Comput..

[15]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[16]  David H. Wolpert,et al.  Bandit problems and the exploration/exploitation tradeoff , 1998, IEEE Trans. Evol. Comput..

[17]  T. Vincent,et al.  A G-function approach to fitness minima, fitness maxima, evolutionarily stable strategies and adaptive landscapes , 1999 .

[18]  D. WhitleyComputer.  A Free Lunch Proof for Gray versus Binary Encodings , 1999 .

[19]  David B. Fogel,et al.  Evolution, neural networks, games, and intelligence , 1999, Proc. IEEE.

[20]  T. M. English.  Optimization is easy and learning is hard in the typical function , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[21]  Jordan B. Pollack,et al.  A Game-Theoretic Approach to the Simple Coevolutionary Algorithm , 2000, PPSN.

[22]  P. Salamon,et al.  Best possible strategy for finding ground states. , 2001, Physical review letters.

[23]  Steffen Christensen,et al.  What can we learn from No Free Lunch? a first attempt to characterize the concept of a searchable function , 2001 .

[24]  David H. Wolpert,et al.  Remarks on a recent paper on the "no free lunch" theorems , 2001, IEEE Trans. Evol. Comput..

[25]  L. D. Whitley,et al.  The No Free Lunch and problem description length , 2001 .

[26]  Thomas Jansen,et al.  Optimization with randomized search heuristics - the (A)NFL theorem, realistic scenarios, and difficult functions , 2002, Theor. Comput. Sci..

[27]  David W. Corne,et al.  No Free Lunch and Free Leftovers Theorems for Multiobjective Optimisation Problems , 2003, EMO.

[28]  Thomas M. English.  On the Structure of Sequential Search: Beyond "No Free Lunch" , 2004, EvoCOP.

[29]  Mihai Oltean,et al.  Searching for a Practical Evidence of the No Free Lunch Theorems , 2004, BioADIT.

[30]  Edwin D. de Jong,et al.  Ideal Evaluation from Coevolution , 2004, Evolutionary Computation.

[31]  D.B. Fogel,et al.  A self-learning evolutionary chess program , 2004, Proceedings of the IEEE.

[32]  Marc Toussaint,et al.  A No-Free-Lunch theorem for non-uniform distributions of target functions , 2004, J. Math. Model. Algorithms.

[33]  Marc Toussaint,et al.  A No-Free-Lunch Theorem for Non-Uniform Distributions of Target Functions , 2004 .

[34]  Michael Wooldridge,et al.  Game Theory and Decision Theory in Multi-Agent Systems , 2002, Autonomous Agents and Multi-Agent Systems.

[35]  J. Pollack,et al.  Order-theoretic Analysis of Coevolution Problems: Coevolutionary Statics , 2007 .