Mechanical versus clinical data combination in selection and admissions decisions: a meta-analysis.

In employee selection and academic admission decisions, holistic (clinical) data combination methods continue to be relied upon and preferred by practitioners in our field. This meta-analysis examined and compared the relative predictive power of mechanical methods versus holistic methods in predicting multiple work (advancement, supervisory ratings of performance, and training performance) and academic (grade point average) criteria. There was consistent and substantial loss of validity when data were combined holistically-even by experts who are knowledgeable about the jobs and organizations in question-across multiple criteria in work and academic settings. In predicting job performance, the difference between the validity of mechanical and holistic data combination methods translated into an improvement in prediction of more than 50%. Implications for evidence-based practice are discussed.

[1]  Simon Haykin,et al.  Cognitive Control , 2012, Proceedings of the IEEE.

[2]  S. McPhail,et al.  Individual Psychological Assessment , 2012 .

[3]  Nathan R. Kuncel,et al.  Some New (and Old) Suggestions for Improving Personnel Selection , 2008, Industrial and Organizational Psychology.

[4]  Scott Highhouse,et al.  Facts Are Stubborn Things , 2008, Industrial and Organizational Psychology.

[5]  Scott Highhouse.  Stubborn Reliance on Intuition and Subjectivity in Employee Selection , 2008, Industrial and Organizational Psychology.

[6]  Gwowen Shieh,et al.  Improved Shrinkage Estimation of Squared Multiple Correlation Coefficient and Squared Cross-Validity Coefficient , 2008 .

[7]  N. Kuncel,et al.  Application patterns when applicants know the odds: implications for selection research and practice. , 2007, The Journal of applied psychology.

[8]  R. Hogarth,et al.  Determinants of Linear Judgment: A Meta-Analysis of Lens Model Studies , 2007, Psychological bulletin.

[9]  Eric Anthony Day,et al.  A META‐ANALYSIS OF THE CRITERION‐RELATED VALIDITY OF ASSESSMENT CENTER DIMENSIONS , 2003 .

[10]  Karen L. Stafford,et al.  Confidence Intervals, Power Calculation, and Sample Size Estimation for the Squared Multiple Correlation Coefficient under the Fixed and Random Regression Models: A Computer Program and Useful Standard Tables , 2001 .

[11]  Xitao Fan,et al.  Estimating R 2 Shrinkage in Multiple Regression: A Comparison of Different Analytical Methods , 2001 .

[12]  Nambury S. Raju,et al.  Accuracy of Population Validity and Cross-Validity Estimation: An Empirical Comparison of Formula-Based, Traditional Empirical, and Equal Weights Procedures , 1999 .

[13]  P. Meehl,et al.  Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. , 1996 .

[14]  Walter C. Borman,et al.  More Progress Toward a Taxonomy of Managerial Performance Requirements , 1993 .

[15]  Rob Feltham,et al.  Assessment centre decision making: judgemental vs. mechanical , 1988 .

[16]  Paul R. Sackett,et al.  A SURVEY OF INDIVIDUAL ASSESSMENT PRACTICES BY I/O PSYCHOLOGISTS , 1987 .

[17]  George C. Thornton,et al.  Meta-analysis of assessment center validity. , 1987 .

[18]  S. Dolan,et al.  Validity of an assessment center for identifying future female officers in the military. , 1982 .

[19]  Walter C. Borman,et al.  Validity of behavioral assessment for predicting military recruiter performance , 1982 .

[20]  Philippe Cattin,et al.  Estimation of the predictive power of a regression model. , 1980 .

[21]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[22]  W W Rozeboom,et al.  Estimation of cross-validated multiple correlation: a clarification. , 1978, Psychological bulletin.

[23]  John G. Claudy.  Multiple Regression and Validity Estimation in One Sample , 1978 .

[24]  James O. Mitchel.  Assessment center validity: A longitudinal study. , 1975 .

[25]  Michael W. Browne,et al.  PREDICTIVE VALIDITY OF A LINEAR REGRESSION EQUATION , 1975 .

[26]  N. Wiggins,et al.  Man versus model of man revisited: The forecasting of graduate school success. , 1971 .

[27]  Karl E. Weick,et al.  Managerial behavior, performance, and effectiveness , 1971 .

[28]  R. Dawes.  A case study of graduate admissions: Application of three principles of human decision making. , 1971 .

[29]  Lewis R. Goldberg,et al.  Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences. , 1970 .

[30]  W. J. Mcnamara,et al.  Relationship of the components of an assessment center to management success. , 1969 .

[31]  Alan Neidich.  Honors Selection Study 1966-67. , 1968 .

[32]  J. Sawyer,et al.  Measurement and prediction, clinical and statistical. , 1966, Psychological bulletin.

[33]  M. Robertson,et al.  PREDICTING SUCCESS IN GRADUATE STUDY. , 1964, The Journal of general psychology.

[34]  Ingram Olkin,et al.  Unbiased Estimation of Certain Correlation Coefficients , 1958 .

[35]  E. Brunswik.  Perception and the Representative Design of Psychological Experiments , 1957 .

[36]  E. Brunswik.  Representative design and probabilistic theory in a functional psychology. , 1955, Psychological review.

[37]  D. B. Stuit.  Personnel research and test development in the Bureau of Naval Personnel , 1947 .

[38]  T. Sarbin.  A Contribution to the Study of Actuarial and Individual Methods of Prediction , 1943, American Journal of Sociology.

[39]  H. Taylor,et al.  The relationship of validity coefficients to the practical effectiveness of tests in selection: discussion and tables. , 1939 .

[40]  R. Wherry,et al.  A New Formula for Predicting the Shrinkage of the Coefficient of Multiple Correlation , 1931 .

[41]  W. Grove,et al.  Clinical versus mechanical prediction: a meta-analysis. , 2000, Psychological assessment.

[42]  R. Fowler.  Confidence intervals for the cross-validated multiple correlation in predictive regression models , 1986 .

[43]  Deane G. Bornheimer.  Predicting Success in Graduate School Using GRE and PAEG Aptitude Test Scores. , 1984 .

[44]  D. Magnusson,et al.  PREDICTIVE EFFICIENCY AS A FUNCTION OF AMOUNT OF INFORMATION. , 1972, Multivariate behavioral research.

[45]  A. C. MacKinney,et al.  Counselor vs. statistical predictions of job satisfaction in engineering. , 1961 .

[46]  J. A. Bath,et al.  Clinical and actuarial predictions of academic survival and attrition. , 1957 .

[47]  John C. Flanagan,et al.  Defining the requirements of the executive's job. , 1951 .