The Supervised Learning No-Free-Lunch Theorems

This paper reviews the supervised learning versions of the no-free-lunch theorems in a simplified form. It also discusses the significance of those theorems, and their relation to other aspects of supervised learning.

[1]  J. Davenport.  Editor , 1960 .

[2]  Temple F. Smith.  Occam's razor , 1980, Nature.

[3]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[4]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .

[5]  Wray L. Buntine,et al.  Bayesian Back-Propagation , 1991, Complex Syst..

[6]  David H. Wolpert,et al.  On the Connection between In-sample Testing and Generalization Error , 1992, Complex Syst..

[7]  David H. Wolpert,et al.  Bayesian Backpropagation Over I-O Functions Rather Than Weights , 1993, NIPS.

[8]  David H. Wolpert,et al.  The Mathematics of Generalization: The Proceedings of the SFI/CNLS Workshop on Formal Approaches to Supervised Learning , 1994 .

[9]  David H. Wolpert,et al.  Filter likelihoods and exhaustive learning , 1994, COLT 1994.

[10]  David H. Wolpert,et al.  The Relationship Between PAC, the Statistical Physics Framework, the Bayesian Framework, and the VC Framework , 1995 .

[11]  Radford M. Neal.  Priors for Infinite Networks , 1996 .

[12]  David H. Wolpert,et al.  The Existence of A Priori Distinctions Between Learning Algorithms , 1996, Neural Computation.

[13]  D. Wolpert.  RECONCILING BAYESIAN AND NON-BAYESIAN ANALYSIS , 1996 .

[14]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[15]  David H. Wolpert,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996, Neural Computation.

[16]  David H. Wolpert,et al.  On Bias Plus Variance , 1997, Neural Computation.

[17]  Russell Greiner,et al.  Computational learning theory and natural learning systems , 1997 .

[18]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .