On the Futility of Blind Search: An Algorithmic View of No Free Lunch

The paper is in three parts. First, we use simple adversary arguments to redevelop and explore some of the no-free-lunch (NFL) theorems and perhaps extend them a little. Second, we clarify the relationship of NFL theorems to algorithm theory and complexity classes such as NP. We claim that NFL is weaker in the sense that the constraints implied by the conjectures of traditional algorithm theory on what an evolutionary algorithm may be expected to accomplish are far more severe than those implied by NFL. Third, we take a brief look at how natural evolution relates to computation and optimization. We suggest that the evolution of complex systems exhibiting high degrees of orderliness is not equivalent in difficulty to optimizing hard (in the complexity sense) problems, and that the optimism in genetic algorithms (GAs) as universal optimizers is not justified by natural evolution. This is an informal tutorial papermost of the information presented is not formally proven, and is either common knowledge or formally proven elsewhere. Some of the claims are intuitions based on experience with algorithms, and in a more formal setting should be classified as conjectures.

[1]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[2]  Kenneth A. De Jong,et al.  Genetic Algorithms are NOT Function Optimizers , 1992, FOGA.

[3]  Joseph C. Culberson,et al.  Mutation-Crossover Isomorphisms and the Construction of Discriminating Functions , 1994, Evolutionary Computation.

[4]  Daniel I. A. Cohen,et al.  Introduction to computer theory , 1986 .

[5]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[6]  J. Culberson.  Iterated Greedy Graph Coloring and the Difficulty Landscape , 1992 .

[7]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[8]  E. Palmer.  Graphical evolution: an introduction to the theory of random graphs , 1985 .

[9]  A. Thomason.  GRAPHICAL EVOLUTION An Introduction to the Theory of Random Graphs (Wiley-Interscience Series in Discrete Mathematics) , 1987 .

[10]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[11]  Jonathan S. Turner,et al.  Almost All k-Colorable Graphs are Easy to Color , 1988, J. Algorithms.

[12]  Gregory J. E. Rawlins.  Proceedings of the First Workshop on Foundations of Genetic Algorithms. Bloomington Campus, Indiana, USA, July 15-18 1990 , 1991, FOGA.

[13]  Michael D. Vose,et al.  Isomorphisms algorithms of genetic , 2002 .

[14]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[15]  William I. Gasarch,et al.  Book Review: An introduction to Kolmogorov Complexity and its Applications Second Edition, 1997 by Ming Li and Paul Vitanyi (Springer (Graduate Text Series)) , 1997, SIGACT News.

[16]  Herbert S. Wilf,et al.  Backtrack: An O(1) Expected Time Algorithm for the Graph Coloring Problem , 1984, Inf. Process. Lett..

[17]  Carsten Lund,et al.  On the hardness of approximating minimization problems , 1993, STOC.

[18]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[19]  S. Kauffman.  At Home in the Universe: The Search for the Laws of Self-Organization and Complexity , 1995 .

[20]  Feng Luo,et al.  Exploring the k-colorable landscape with Iterated Greedy , 1993, Cliques, Coloring, and Satisfiability.

[21]  Bernd Hartke,et al.  Global cluster geometry optimization by a phenotype algorithm with Niches: Location of elusive minima, and low‐order scaling with cluster size , 1999 .

[22]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[23]  Kennetb A. De.  Genetic Algorithms Are NOT Function Optimizers , 1992 .

[24]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[25]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[26]  A. Beacham.  Hiding Our Colors , 1995 .

[27]  Lee Altenberg,et al.  The Schema Theorem and Price's Theorem , 1994, FOGA.

[28]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.