Recent Results on No-Free-Lunch Theorems for Optimization

The sharpened No-Free-Lunch-theorem (NFL-theorem) states that the performance of all optimization algorithms averaged over any finite set F of functions is equal if and only if F is closed under permutation (c.u.p.) and each target function in F is equally likely. In this paper, we first summarize some consequences of this theorem, which have been proven recently: The average number of evaluations needed to find a desirable (e.g., optimal) solution can be calculated; the number of subsets c.u.p. can be neglected compared to the overall number of possible subsets; and problem classes relevant in practice are not likely to be c.u.p. Second, as the main result, the NFL-theorem is extended. Necessary and sufficient conditions for NFL-results to hold are given for arbitrary, non-uniform distributions of target functions. This yields the most general NFL-theorem for optimization presented so far.

[1]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[2]  T. M. English Optimization is easy and learning is hard in the typical function , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[3]  David H. Wolpert,et al.  Remarks on a recent paper on the "no free lunch" theorems , 2001, IEEE Trans. Evol. Comput..

[4]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[5]  Christian Igel,et al.  Graph isomorphisms effect on structure optimization of neural networks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[6]  Marc Toussaint,et al.  Neutrality and self-adaptation , 2003, Natural Computing.

[7]  D. WhitleyComputer A Free Lunch Proof for Gray versus Binary Encodings , 1999 .

[8]  Thomas Jansen,et al.  Optimization with randomized search heuristics - the (A)NFL theorem, realistic scenarios, and difficult functions , 2002, Theor. Comput. Sci..

[9]  Marc Toussaint,et al.  On Classes of Functions for which No Free Lunch Results Hold , 2001, Inf. Process. Lett..

[10]  Patrick D. Surry,et al.  Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective , 1995, Computer Science Today.