Model Selection Via Multifold Cross Validation

In model selection, it is known that the simple leave one out cross validation method is apt to select overfitted models. In an attempt to remedy this problem, we consider two notions of multi-fold cross validation (MCV and MCV*) criteria. In the case of linear regression models, their performance is studied and compared with the simple CV method. As expected, it turns out that MCV indeed reduces the chance of overfitting. The intent ofMCV' is rather different from that of MCV. The differences between these two notions of MCV are also discussed. Our result explains the phenomena observed by Breiman & Spector.