A COMPARISON OF CRITERIA FOR TEST VALIDATION: A META‐ANALYTIC INVESTIGATION

Meta-analyses of validity coefficients from tests of clerical abilities for five criteria—supervisor ratings, supervisor rankings, work samples, production quantity, and production quality—were conducted, and the resulting expected true validities were compared. Ratings, rankings, work samples, and production quantity all resulted in high test validities. Validities resulting from ratings and quantity-of-production criteria were highly similar across tests. Validities resulting from rankings and work samples were on the average higher than those from ratings and quantity of production. The fifth criterion, quality of production, had low predictability and did not generalize across situations.

[1]  D. Kipnis,et al.  Why do I like thee: Is it your performance or my orders? , 1981 .

[2]  Paul W. Paese,et al.  Validity generalization and hypothetical reliability distributions: A test of the Schmidt-Hunter procedure. , 1988 .

[3]  J. W. Tilton.  The measurement of overlapping. , 1937 .

[4]  J. Hunter,et al.  Validity and Utility of Alternative Predictors of Job Performance , 1984 .

[5]  Michael J. Kavanagh,et al.  Issues in managerial performance: Multitrait-multimethod analyses of ratings. , 1971 .

[6]  D. L. Grant.  A factor analysis of managers' ratings. , 1955 .

[7]  F. Schmidt,et al.  Fairness of psychological tests: Implications of four definitions for selection utility and minority hiring. , 1977 .

[8]  James M. Price,et al.  Consistency of performance under stable conditions of work. , 1983 .

[9]  Michael P. Kirsch,et al.  METAANALYSES OF VALIDITY STUDIES PUBLISHED BETWEEN 1964 AND 1982 AND THE INVESTIGATION OF STUDY CHARACTERISTICS , 1984 .

[10]  John E. Hunter,et al.  Validity generalization results for computer programmers. , 1980 .

[11]  Edward R. Kemery,et al.  The power of the Schmidt and Hunter Additive Model of Validity Generalization. , 1987 .

[12]  Bruce J. Avolio,et al.  A meta-analysis of age differences in job performance. , 1986 .

[13]  Hubert E. Brogden,et al.  The Theory and Classification of Criterion Bias , 1950 .

[14]  R. Boruch,et al.  Alternative Methods of Analysis: Multitrait-Multimethod Data , 1970 .

[15]  W. Edwards Deming,et al.  On Some Statistical Aids Toward Economic Production , 1975 .

[16]  Ralph A. Alexander,et al.  An Application of Meta-Analysis to Theory Building and Construct Validation. , 1985 .

[17]  J. A. Creager,et al.  A hierarchical factor analysis of foreman behavior. , 1958 .

[18]  Jack M. Feldman,et al.  Beyond Attribution Theory: Cognitive Processes in Performance Appraisal , 1981 .

[19]  E. Lawler,et al.  The multitrait-multirater approach to measuring managerial job performance. , 1967, The Journal of applied psychology.

[20]  H. F. Rothe,et al.  Output rates among industrial employees. , 1978 .

[21]  John E. Hunter,et al.  Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. , 1980 .

[22]  Robert G. Lord,et al.  Cognitive categorization and dimensional schemata: A process approach to the study of halo in performance ratings. , 1983 .

[23]  Robert L. Holzbach,et al.  Rater bias in performance ratings: Superior, self-, and peer ratings. , 1978 .

[24]  C. I. Mosier.  I. Problems and Designs of Cross-Validation 1 , 1951 .

[25]  Paul R. Sackett,et al.  On seeking moderator variables in the meta-analysis of correlational data: A Monte Carlo investigation of statistical power and resistance to Type I error. , 1986 .

[26]  D. Rubin,et al.  Comparing Effect Sizes of Independent Studies , 1982 .

[27]  J. Hunter.  Cognitive ability, cognitive aptitudes, job knowledge, and job performance , 1986 .

[28]  Paul E. Spector,et al.  Unemployment, job satisfaction, and employee turnover: A meta-analytic test of the Muchinsky model. , 1987 .

[29]  K. Kraiger,et al.  Study of race effects in objective indices and subjective evaluations of performance: A meta-analysis of performance criteria. , 1986 .

[30]  H. Aurbach,et al.  PREDICTORS, CRITERIA, AND SIGNIFICANT RESULTS , 1971 .

[31]  R. Alexander,et al.  The role of inferential accuracy in performance rating. , 1985, Academy of management review. Academy of Management.

[32]  F. Schmidt,et al.  VALIDITY GENERALIZATION RESULTS FOR LAW ENFORCEMENT OCCUPATIONS , 1986 .

[33]  R. Reilly,et al.  VALIDITY AND FAIRNESS OF SOME ALTERNATIVE EMPLOYEE SELECTION PROCEDURES , 1982 .

[34]  Frank J. Landy,et al.  Statistical control of halo error in performance ratings. , 1980 .

[35]  Ervin W. Curtis,et al.  A correlational approach for measuring overlap of distributions and proportion of misclassification. , 1968 .