Cross-Validation for Prediction

The authors review and critically evaluate three major approaches to cross-validation in the context of predictive validity: (1) sample-splitting, (2) resampling plans such as bootstrapping, and (3) a method that simultaneously estimates parameters and cross-validates. Because of the information loss involved and the availability of shrinkage formulas, they argue that data-splitting is inferior to the simultaneous approach. An empirical example illustrates how the simultaneous approach can be used in conjunction with bootstrap resampling to construct prediction intervals that are superior to classical prediction intervals.

[1]  B. Efron.  The jackknife, the bootstrap, and other resampling plans , 1987 .

[2]  Seymour Geisser,et al.  The Predictive Sample Reuse Method with Applications , 1975 .

[3]  M. R. Mickey,et al.  Estimation of Error Rates in Discriminant Analysis , 1968 .

[4]  William D. Perreault,et al.  Validation of Discriminant Analysis in Marketing Research , 1977 .

[5]  Paul Horst,et al.  The prediction of personal adjustment. , 1942 .

[6]  M. Stone.  Asymptotics for and against cross-validation , 1977 .

[7]  R. Stine.  Bootstrap Prediction Intervals for Regression , 1985 .

[8]  Philippe Cattin,et al.  On the Use of Formulas of the Predictive Validity of Regression in Consumer Research , 1979 .

[9]  R. Dennis Cook,et al.  Cross-Validation of Regression Models , 1984 .

[10]  B. Efron.  Estimating the Error Rate of a Prediction Rule: Improvement on Cross-Validation , 1983 .

[11]  Ronald D. Snee,et al.  Validation of Regression Models: Methods and Examples , 1977 .

[12]  M. Stone,et al.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[13]  Ian Fenwick,et al.  Techniques in Market Measurement: The Jackknife , 1979 .

[14]  Neil J. Dorans,et al.  A note on cross-validating prediction equations , 1980 .

[15]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[16]  C. I. Mosier.  I. Problems and Designs of Cross-Validation 1 , 1951 .

[17]  D. Freedman,et al.  How Many Variables Should Be Entered in a Regression Equation , 1983 .

[18]  Lawrence R. James,et al.  A Note on Validity Generalization Procedures. , 1986 .

[19]  S. Larson.  The shrinkage of the coefficient of multiple correlation. , 1931 .

[20]  M. Stone.  An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[21]  R. Shibata.  An optimal selection of regression variables , 1981 .

[22]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[23]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[24]  C. B. Tilanus,et al.  Applied Economic Forecasting , 1966 .

[25]  Kevin R. Murphy,et al.  FOOLING YOURSELF WITH CROSS‐VALIDATION: SINGLE SAMPLE DESIGNS , 1983 .

[26]  W. Mendenhall,et al.  A second course in business statistics: Regression analysis , 1981 .

[27]  D. Basu.  Randomization Analysis of Experimental Data: The Fisher Randomization Test , 1980 .

[28]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[29]  Philip J. McCarthy,et al.  The Use of Balanced Half-Sample Replication in Cross-Validation Studies , 1976 .

[30]  R. Darlington,et al.  Multiple regression in psychological research and practice. , 1968, Psychological bulletin.

[31]  Frederick Mosteller,et al.  Data Analysis and Regression , 1978 .

[32]  J. Elashoff,et al.  Multiple Regression in Behavioral Research. , 1975 .

[33]  Philippe Cattin,et al.  A Simple Bayesian Procedure for Estimation in a Conjoint Model , 1983 .