An Initial Validation of Developmental Assessment Centers as Accurate Assessments and Effective Training Interventions

This article takes a critical look at the various forms of validity evidence that can be marshaled to support the effectiveness of a developmental assessment center (DAC) program. Using the holistic view of validity (American Educational Research Association, American Psychological Association, & American Council on Measurement in Education, 1999; Binning & Barrett, 1989; Messick, 1998; Society for Industrial and Organizational Psychology, 2003) as well as Kirkpatrick's (1976) taxonomy of training outcomes, we discuss how DACs can be validated in terms of both their assessment accuracy and their effectiveness as training interventions. Types of evidence include appropriate content; internal structure; relationships with other variables; social consequences; and reactions, learning, behavioral, and results criteria. Evidence from many of these categories is then provided from two operational DACs. Results indicate that whereas DACs may be useful tools for developing managerial competencies, continued resea...

[1]  Filip Lievens,et al.  Large-scale investigation of the role of trait activation theory for understanding assessment center convergent and discriminant validity. , 2006, The Journal of applied psychology.

[2]  James W. Smither,et al.  DOES PERFORMANCE IMPROVE FOLLOWING MULTISOURCE FEEDBACK? A THEORETICAL MODEL, META‐ANALYSIS, AND REVIEW OF EMPIRICAL FINDINGS , 2005 .

[3]  Michael P. Clements.  Decision-based Evaluation , 2005 .

[4]  Filip Lievens,et al.  Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings. , 2004, The Journal of applied psychology.

[5]  Eric Anthony Day,et al.  A META‐ANALYSIS OF THE CRITERION‐RELATED VALIDITY OF ASSESSMENT CENTER DIMENSIONS , 2003 .

[6]  F. Lievens.  Trying to understand the different pieces of the construct validity puzzle of assessment centers: an examination of assessor and assessee effects. , 2002, The Journal of applied psychology.

[7]  Deidra J. Schleicher,et al.  A new frame for frame-of-reference training: enhancing the construct validity of assessment centers. , 2002, The Journal of applied psychology.

[8]  N. Christiansen,et al.  IMPLICATIONS OF TRAIT‐ACTIVATION THEORY FOR EVALUATING THE CONSTRUCT VALIDITY OF ASSESSMENT CENTER RATINGS , 2002 .

[9]  Marylène Gagné,et al.  Pursuing academic interests: The role of implicit theories. , 2001 .

[10]  Donald E. Conlon,et al.  Justice at the millennium: a meta-analytic review of 25 years of organizational justice research. , 2001, The Journal of applied psychology.

[11]  F. Lievens,et al.  Assessor training strategies and their effects on accuracy, interrater reliability, and discriminant validity. , 2001, The Journal of applied psychology.

[12]  Dharm P. S. Bhawuk,et al.  Evolution of culture assimilators , 2001 .

[13]  Sarah A. Hezlett,et al.  Individual Development in the Workplace , 2001 .

[14]  Charles E. Lance,et al.  Assessment Center Exercise Factors Represent Cross-Situational Specificity, Not Method Bias , 2000 .

[15]  Dennis A. Joiner.  Guidelines and Ethical Considerations for Assessment Center Operations , 2000 .

[16]  R. Tett,et al.  Development and Content Validation of a "Hyperdimensional" Taxonomy of Managerial Competence , 2000 .

[17]  Richard S. Williams,et al.  Development centres‐ a review of assumptions , 1999 .

[18]  S. Messick.  Test Validity: A Matter of Consequence , 1998 .

[19]  H. Tillema.  Assessment of Potential, from Assessment Centers to Development Centers , 1998 .

[20]  F. Schmidt,et al.  The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. , 1998 .

[21]  M. Fulford,et al.  Age Bias in Assessment Center Ratings , 1997 .

[22]  Miguel A. Quiñones,et al.  A survey of assessment center practices in organizations in the United States. , 1997 .

[23]  Alex Howard,et al.  A reassessment of assessment centers: Challenges for the 21st century , 1997 .

[24]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[25]  David Dunning,et al.  Trait Importance and Modifiability as Factors Influencing Self-Assessment and Self-Enhancement Motives , 1995 .

[26]  Robert G. Jones,et al.  EVALUATING DEVELOPMENTAL ASSESSMENT CENTERS AS INTERVENTIONS , 1995 .

[27]  A. H. Fischer,et al.  The Managerial Performance Implications of a Developmental Assessment Center Process , 1995 .

[28]  Kevin R. Murphy,et al.  Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives , 1995 .

[29]  E. Salas,et al.  Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. , 1993 .

[30]  Martha Reeves,et al.  Evaluation of Training , 1993 .

[31]  Walter C. Borman,et al.  More Progress Toward a Taxonomy of Managerial Performance Requirements , 1993 .

[32]  George C. Thornton,et al.  Construct Validity of Self- and Peer Evaluations of Performance Dimensions in an Assessment Center , 1992 .

[33]  George C. Thornton,et al.  Assessment Centers in Human Resource Management , 1992 .

[34]  W. B. Johnston,et al.  Global work force 2000: the new world labor market. , 1991, Harvard business review.

[35]  L. Shore,et al.  CONSTRUCT VALIDITY OF TWO CATEGORIES OF ASSESSMENT CENTER DIMENSION RATINGS , 1990 .

[36]  Gerald V. Barrett,et al.  Validity of Personnel Decisions: A Conceptual Analysis of the Inferential and Evidential Bases , 1989 .

[37]  G. Alliger,et al.  KIRKPATRICK'S LEVELS OF TRAINING CRITERIA: THIRTY YEARS LATER , 1989 .

[38]  Lee J. Cronbach,et al.  Construct validation after thirty years. , 1989 .

[39]  C. Dweck,et al.  A social-cognitive approach to motivation and personality , 1988 .

[40]  Michael J. Kavanagh,et al.  Improving the accuracy of performance evaluations: comparison of three methods of performance appraiser training , 1988 .

[41]  George C. Thornton,et al.  Meta-analysis of assessment center validity. , 1987 .

[42]  D. Cochran,et al.  Designing a Developmental Assessment Center in a Government Agency: A Case Study , 1987 .

[43]  Elaine D. Pulakos,et al.  A comparison of rater training programs: Error training and accuracy training. , 1984 .

[44]  Betty Jo Licata,et al.  The Impact of Skill Learnability on the Effectiveness of Managerial Training and Development , 1983 .

[45]  Wayne F. Cascio,et al.  Performance Appraisal Decisions as a Function of Rater Training and Purpose of the Appraisal , 1982 .

[46]  Susan E. Jackson,et al.  Explaining performance variability: Contributions of goal setting, task characteristics, and evaluative contexts. , 1982 .

[47]  Clive Fletcher,et al.  The relationship between previous experience, intelligence and background characteristics of participants and their performance in an assessment centre , 1982 .

[48]  C G Thornton,et al.  Assessment Center and managerial performance , 1982 .

[49]  P. Sackett.  A Critical Look at Some Common Beliefs about Assessment Centers , 1982 .

[50]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[51]  James A. Waters,et al.  Managerial Skill Development , 1980 .

[52]  Douglas W. Bray,et al.  Management Assessment Center Evaluations and Subsequent Job Performance of White and Black Females. , 1976 .

[53]  Joseph L. Moses,et al.  Relationship of assessment-center performance to management progress of women. , 1975 .

[54]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[55]  D. Bray,et al.  The assessment center in the measurement of potential for business management. , 1966, Psychological monographs.