Note on Free Lunches and Cross-Validation

The no-free-lunch theorems (Wolpert & Macready, 1995) have sparked heated debate in the computational learning community. A recent communication (Zhu & Rohwer, 1996) attempts to demonstrate the inefficiency of cross-validation on a simple problem. We elaborate on this result by considering a broader class of cross-validation. When used more strictly, cross-validation can yield the expected results on simple examples.

[1]  Huaiyu Zhu,et al.  No Free Lunch for Cross-Validation , 1996, Neural Computation.

[2]  Malik Magdon-Ismail,et al.  No Free Lunch for Early Stopping , 1999, Neural Computation.

[3]  Léon Personnaz,et al.  On Cross Validation for Model Selection , 1999, Neural Computation.

[4]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[5]  David H. Wolpert,et al.  The Mathematics of Search , 1996 .