FOOLING YOURSELF WITH CROSS‐VALIDATION: SINGLE SAMPLE DESIGNS

The most commonly used cross-validation design involves drawing a single sample and partitioning that sample into derivation and holdout subsamples. This type of design allows one to adjust for random sampling error, but like formula estimates of cross-validity, is insensitive to violations of sampling assumptions. As is shown in a small Monte Carlo study, results obtained in non-representative samples, which are known to be invalid in the population, will nonetheless hold up well under cross-validation when single-sample designs are employed. It is suggested that single-sample cross-validation estimates possess no clear-cut advantages over formula estimates, and thus are not worth the effort or the loss of degrees of freedom.

[1]  R. M. Thorndike,et al.  Correlational Procedures for Research , 1979 .

[2]  Neil J. Dorans,et al.  Estimators of the Squared Cross-Validity Coefficient: A Monte Carlo Investigation , 1979 .

[3]  R. Oliver,et al.  PREDICTING TENURE WITH BIOGRAPHICAL DATA: EXHUMING BURIED EVIDENCE , 1974 .

[4]  E. Alf,et al.  A SEX COMPARISON OF FACTORS RELATED TO SUCCESS IN NAVAL HOSPITAL CORPS SCHOOL , 1978 .

[5]  J. Elashoff,et al.  Multiple Regression in Behavioral Research. , 1975 .

[6]  Richard J. Harris.  A primer of multivariate statistics , 1975 .

[7]  Bernard M. Bass,et al.  Lifespace variables and managerial success. , 1978 .

[8]  R. L. Horton.  The General Linear Model: Data Analysis in the Social and Behavioral Sciences. , 1987 .

[9]  N. H. Berry,et al.  PREDICTING PERFORMANCE EFFECTIVENESS IN PARAMEDICAL OCCUPATIONS , 1978 .

[10]  Neal Schmitt,et al.  A Monte Carlo evaluation of three formula estimates of cross-validated multiple correlation. , 1977 .

[11]  John G. Claudy.  Multiple Regression and Validity Estimation in One Sample , 1978 .

[12]  C. Pinder.  MULTIPLE PREDICTORS OF POST‐TRANSFER SATISFACTION: THE ROLE OF URBAN FACTORS , 1977 .

[13]  Marvin D. Dunnette,et al.  Personnel Selection and Placement , 1966 .

[14]  G. A. Colbert,et al.  EMPIRICALLY DERIVED JOB FAMILIES AS A FOUNDATION FOR THE STUDY OF VALIDITY GENERALIZATION.: STUDY III. GENERALIZATION OF SELECTION TEST VALIDITY , 1978 .

[15]  Neal Schmitt,et al.  Applicant decisions in the employment interview. , 1976 .

[16]  Wayne F. Cascio,et al.  Relations among criteria of police performance. , 1978 .

[17]  Donald P. Schwab,et al.  A cross-validated comparison of five models used to predict graduate admissions committee decisions. , 1976 .

[18]  Kevin R. Murphy,et al.  Correlates of perceived fairness and accuracy of performance evaluation. , 1978 .

[19]  R. Darlington,et al.  Multiple regression in psychological research and practice. , 1968, Psychological bulletin.

[20]  William M. Pugh,et al.  Identifying Determinants of Retention Decisions. , 1977 .

[21]  L. R. Taylor.  EMPIRICALLY DERIVED JOB FAMILIES AS A FOUNDATION FOR THE STUDY OF VALIDITY GENERALIZATION , 1978 .

[22]  F. E. Saal,et al.  Job involvement: A multivariate approach. , 1978 .

[23]  Gary P. Latham,et al.  BEHAVIORAL OBSERVATION SCALES FOR PERFORMANCE APPRAISAL PURPOSES , 1977 .