A META‐ANALYSIS OF THE CRITERION‐RELATED VALIDITY OF ASSESSMENT CENTER DIMENSIONS

We used meta-analytic procedures to investigate the criterion-related validity of assessment center dimension ratings. By focusing on dimension-level information, we were able to assess the extent to which specific constructs account for the criterion-related validity of assessment centers. From a total of 34 articles that reported dimension-level validities, we collapsed 168 assessment center dimension labels into an overriding set of 6 dimensions: (a) consideration/awareness of others, (b) communication, (c) drive, (d) influencing others, (e) organizing and planning, and (f) problem solving. Based on this set of 6 dimensions, we extracted 258 independent data points. Results showed a range of estimated true criterion-related validities from .25 to .39. A regression-based composite consisting of 4 out of the 6 dimensions accounted for the criterion-related validity of assessment center ratings and explained more variance in performance (20%) than Gaugler, Rosenthal, Thornton, and Bentson (1987) were able to explain using the overall assessment center rating (14%).

[1]  Gerald V. Barrett,et al.  Validity of Personnel Decisions: A Conceptual Analysis of the Inferential and Evidential Bases , 1989 .

[2]  James O. Mitchel.  Assessment center validity: A longitudinal study. , 1975 .

[3]  F. Lievens.  Assessors and use of assessment centre dimensions: a fresh look at a troubling issue* , 2001 .

[4]  James W. Thacker,et al.  Self-monitoring and Sex As Related to Assessment Center Rating and Job Performance , 1985 .

[5]  J. Conway,et al.  Dimension and exercise variance in assessment center scores: a large-scale evaluation of multitrait-multimethod studies. , 2001, The Journal of applied psychology.

[6]  R. Goffin,et al.  Personality testing and the assessment center : Incremental validity for managerial selection , 1996 .

[7]  Neal Schmitt,et al.  Validity of assessment center ratings for the prediction of performance ratings and school climate of school administrators. , 1984 .

[8]  C. Russell,et al.  Two field tests of an explanation of assessment centre validity , 1995 .

[9]  H. G. Osburn,et al.  Effects of the Rating Process on the Construct Validity of Assessment Center Dimension Evaluations , 2000 .

[10]  N. Schmitt,et al.  An exercise design approach to understanding assessment center dimension and exercise constructs. , 1992 .

[11]  Richard J. Klimoski,et al.  WHY DO ASSESSMENT CENTERS WORK? THE PUZZLE OF ASSESSMENT CENTER VALIDITY , 1987 .

[12]  George C. Thornton,et al.  Meta-analysis of assessment center validity. , 1987 .

[13]  C. Russell.  Individual Decision Processes in an Assessment Center , 1985 .

[14]  George C. Thornton,et al.  Construct Validity of Self- and Peer Evaluations of Performance Dimensions in an Assessment Center , 1992 .

[15]  Michael M. Harris,et al.  The Measurement of Assessment Center Situations: Bem's Template Matching Technique for Examining Exercise Similarity , 1993 .

[16]  Allen I. Huffcutt,et al.  Development of a new outlier statistic for meta-analytic data. , 1995 .

[17]  P. Bycio,et al.  Situational specificity in assessment center ratings: A confirmatory factor analysis. , 1987 .

[18]  G. Dobbins,et al.  New Evidence On the Construct Validity of Diagnostic Assessment Centers: The Findings May Not Be So Troubling After All , 1997 .

[19]  A. Huffcutt,et al.  Identification and meta-analytic assessment of psychological constructs measured in employment interviews. , 2001 .

[20]  M. Hakel,et al.  Temporal stability and individual differences in using assessment information to form overall ratings , 1979 .

[21]  Robert G. Jones,et al.  EVALUATING DEVELOPMENTAL ASSESSMENT CENTERS AS INTERVENTIONS , 1995 .

[22]  F. Lievens.  Factors which Improve the Construct Validity of Assessment Centers: A Review , 1998 .

[23]  R. Ritchie,et al.  Assessment center correlates of women's advancement into middle management: A 7-year longitudinal analysis. , 1983 .

[24]  Joseph L. Moses,et al.  Relationship of assessment-center performance to management progress of women. , 1975 .

[25]  Miguel A. Quiñones,et al.  A survey of assessment center practices in organizations in the United States. , 1997 .

[26]  Edwin E. Ghiselli,et al.  THE VALIDITY OF APTITUDE TESTS IN PERSONNEL SELECTION , 1973 .

[27]  George C. Thornton,et al.  Number of assessment center dimensions as a determinant of assessor accuracy. , 1989 .

[28]  H. A. Thomson.  Comparison of predictor and criterion judgments of managerial performance using the multitrait-multimethod approach. , 1970 .

[29]  F. Schmidt,et al.  The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. , 1998 .

[30]  T. Garavan,et al.  Graduate assessment centres: An empirical investigation of effectiveness. , 1998 .

[31]  Bryan D. Edwards,et al.  MULTIPLE‐CHOICE AND CONSTRUCTED RESPONSE TESTS OF ABILITY: RACE‐BASED SUBGROUP PERFORMANCE DIFFERENCES ON ALTERNATIVE PAPER‐AND‐PENCIL TEST FORMATS , 2002 .

[32]  Robert G. Jones.  Construct validation of assessment center final dimension ratings: Definition and measurement issues , 1992 .

[33]  J. Hunter,et al.  Validity and Utility of Alternative Predictors of Job Performance , 1984 .

[34]  Frederick O. Carleton.  Relationships between follow-up evaluations and information developed in a management assessment center. , 1970 .

[35]  John W. Fleenor.  Constructs and developmental assessment centers: Further troubling empirical findings , 1996 .

[36]  T. C. Parker.  Assessment Centers: A Statistical Study. , 1980 .

[37]  Deidra J. Schleicher,et al.  A new frame for frame-of-reference training: enhancing the construct validity of assessment centers. , 2002, The Journal of applied psychology.

[38]  D. Ones,et al.  Theory testing: Combining psychometric meta-analysis and structural equations modeling , 1995 .

[39]  Janet J. Turnage,et al.  Transsituational variability in human performance within assessment centers , 1982 .

[40]  Murray R. Barrick,et al.  THE BIG FIVE PERSONALITY DIMENSIONS AND JOB PERFORMANCE: A META-ANALYSIS , 1991 .

[41]  Dennis Doverspike,et al.  DEVELOPMENT OF A JOB ANALYSIS-BASED PROCEDURE FOR WEIGHTING AND COMBINING CONTENT RELATED TESTS INTO A SINGLE TEST BATTERY SCORE , 1996 .

[42]  Michael P. Kirsch,et al.  METAANALYSES OF VALIDITY STUDIES PUBLISHED BETWEEN 1964 AND 1982 AND THE INVESTIGATION OF STUDY CHARACTERISTICS , 1984 .

[43]  W. Dunlap,et al.  On the bias of Huffcutt and Arthur's (1995) procedure for identifying outliers in the meta-analysis of correlations. , 2002, Journal of Applied Psychology.

[44]  A. Dalessio,et al.  INFLUENCE OF ASSESSMENT CENTER METHODS ON ASSESSORS’RATINGS , 1986 .

[45]  R. Finley.  Evaluation of Behavior Predictions from Projective Tests Given in a Management Assessment Center: (465412008-039) , 1970 .

[46]  John R. Hinrichs,et al.  An Eight-Year Follow-Up of a Management Assessment Center. , 1978 .

[47]  F. Lievens.  Trying to understand the different pieces of the construct validity puzzle of assessment centers: an examination of assessor and assessee effects. , 2002, The Journal of applied psychology.

[48]  R. Reilly,et al.  VALIDITY AND FAIRNESS OF SOME ALTERNATIVE EMPLOYEE SELECTION PROCEDURES , 1982 .

[49]  Gregory M. Hurtz,et al.  Personality and job performance: the Big Five revisited. , 2000, The Journal of applied psychology.

[50]  Paul R. Sackett,et al.  Constructs and assessment center dimensions: Some troubling empirical findings , 1982 .

[51]  Ted H. Shore.  Subtle gender bias in the assessment of managerial potential , 1992 .

[52]  A. Sagie,et al.  Assessor type, number of distinguishable dimension categories, and assessment centre construct validity , 1997 .

[53]  N. Schmitt,et al.  Video-based versus paper-and-pencil method of assessment in situational judgment tests: subgroup differences in test performance and face validity perceptions. , 1997, The Journal of applied psychology.

[54]  W. Graziano,et al.  The five-factor model, conscientiousness, and driving accident involvement. , 1996, Journal of personality.

[55]  W. J. Mcnamara,et al.  Relationship of the components of an assessment center to management success. , 1969 .

[56]  Ellen M. Whitener,et al.  Confusion of confidence intervals and credibility intervals in meta-analysis. , 1990 .

[57]  Paul R. Sackett,et al.  ASSESSMENT CENTERS AND CONTENT VALIDITY: SOME NEGLECTED ISSUES , 1987 .

[58]  Winfred Arthur,et al.  Convergent and Discriminant Validity of Assessment Center Dimensions: A Conceptual and Empirical Reexamination of the Assessment Center Construct-Related Validity Paradox , 2000 .

[59]  Using the assessment center method to predict senior management potential. , 1994 .

[60]  Alan Jones,et al.  Attempting to improve the validity of a well-established assessment centre , 1991 .