The Knowledge, Skill, and Ability Requirements for Teamwork: Revisiting the Teamwork‐KSA Test's Validity

The Teamwork – Knowledge, Skills, and Ability (KSA) Test was developed by Stevens and Campion to operationalize their comprehensive taxonomy of teamwork competencies. The test is generally considered ‘valid’ and has been used frequently in organizations. Our review of the literature found an average criterion validity of.20 for the Teamwork‐KSA Test, although there was considerable variability across studies. We could find no research on the item properties, factor structure, or subscale reliabilities, and no extensive investigations of the nomological net of this test. In our field sample, we found subscale reliabilities to be generally inadequate, no meaningful factor structure, and low predictivenes of employees' performance on team‐related dimensions. Although the taxonomy it purports to measure is preeminent, the Teamwork‐KSA Test itself may have serious limitations.

[1]  Daniel A. Newman,et al.  Emotional intelligence: an integrative meta-analysis and cascading model. , 2010, The Journal of applied psychology.

[2]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[3]  Thomas R. Carretta,et al.  Emotional intelligence: not much more than g and personality , 2004 .

[4]  Tammy L. Rapp,et al.  Evaluating an Individually Self-Administered Generic Teamwork Skills Training Program Across Time and Levels , 2007 .

[5]  J. Kottke,et al.  Able But Not Willing? Teamwork Aptitude and Interest Meet Head-on , 2008 .

[6]  R. Goffin,et al.  Taking advantage of social comparisons in performance appraisal: The relative percentile method , 2009 .

[7]  Wayne F. Velicer,et al.  Construct Explication through Factor or Component Analysis: A Review and Evaluation of Alternative Procedures for Determining the Number of Factors or Components , 2000 .

[8]  Paul Kline,et al.  A Handbook of Test Construction : Introduction to Psychometric Design , 1987 .

[9]  Samuel Messick,et al.  VALIDITY OF TEST INTERPRETATION AND USE , 1990 .

[10]  Michael A. West,et al.  Selecting for teamwork , 1997 .

[11]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[12]  Murray R. Barrick,et al.  Five-Factor Model of personality and Performance in Jobs Involving Interpersonal Interactions , 1998 .

[13]  Ian R. Gellatly,et al.  Test-Taking Motivation and Personality Test Validity , 2010 .

[14]  F. Morgeson,et al.  SELECTING INDIVIDUALS IN TEAM SETTINGS: THE IMPORTANCE OF SOCIAL SKILLS, PERSONALITY CHARACTERISTICS, AND TEAMWORK KNOWLEDGE , 2005 .

[15]  Paul R. Jackson,et al.  Team Autonomy, Performance, and Member Job Strain: Uncovering the Teamwork KSA Link , 2005 .

[16]  Ann Marie Ryan,et al.  Test-taking dispositions: A missing link? , 1992 .

[17]  Elena Maceviciute,et al.  Review of: Einspruch, Eric L. An introductory guide to SPSS® for Windows®. 2nd ed. Thousand Oaks, CA: Sage Publications, 2005 , 2006, Inf. Res..

[18]  Ronald F. Piccolo,et al.  A META‐ANALYSIS OF TEAMWORK PROCESSES: TESTS OF A MULTIDIMENSIONAL MODEL AND RELATIONSHIPS WITH TEAM EFFECTIVENESS CRITERIA , 2008 .

[19]  Chockalingam Viswesvaran,et al.  Personality and absenteeism: a meta‐analysis of integrity tests , 2003 .

[20]  R. Goffin,et al.  Selection Test Anxiety: Exploring Tension and Fear of Failure Across the Sexes in Simulated Selection Scenarios , 2005 .

[21]  L. Morris,et al.  Cognitive and Emotional Components of Test Anxiety: A Distinction and Some Initial Data , 1967, Psychological reports.

[22]  Christine A. Henle,et al.  Measurement in Work and Organizational Psychology , 2001 .

[23]  P. Bobko.  Correlation and Regression: Applications for Industrial Organizational Psychology and Management , 2001 .

[24]  Diane L. Miller,et al.  Reexamining Teamwork KSAs and Team Performance , 2001 .

[25]  Thomas G Rundall,et al.  Evidence-based management. , 2007, Hospitals & health networks.

[26]  Robert M. Guion,et al.  Assessment, Measurement, and Prediction for Personnel Decisions , 1997 .

[27]  Robert R. Hirschfeld,et al.  Becoming team players: team members' mastery of teamwork knowledge as a predictor of team task proficiency and observed teamwork effectiveness. , 2006, The Journal of applied psychology.

[28]  Chockalingam Viswesvaran,et al.  Comparative analysis of the reliability of job performance ratings , 1996 .

[29]  T. O’Neill,et al.  Explaining workplace deviance behavior with more than just the “Big Five” , 2011 .

[30]  R. Goffin,et al.  Is the Test Attitude Survey Psychometrically Sound? , 2003 .

[31]  Stephen H. Wagner,et al.  Differences in Accuracy of Absolute and Comparative Performance Appraisal Methods , 1997 .

[32]  Denise M. Rousseau,et al.  Is there Such a thing as “Evidence-Based Management”? , 2006 .

[33]  R. Goffin.  A Comparison of Two New Indices for the Assessment of Fit of Structural Equation Models , 1993 .

[34]  D. Ones,et al.  Measurement Error in “Big Five Factors” Personality Assessment: Reliability Generalization across Studies and Measures , 2000 .

[35]  D. Paulhus.  Measurement and control of response bias. , 1991 .

[36]  Michael J. Stevens,et al.  Staffing Work Teams: Development and Validation of a Selection Test for Teamwork Settings , 1999 .

[37]  J. Loevinger.  Objective Tests as Instruments of Psychological Theory , 1957 .

[38]  P. Bentler.  Comparative Fit Indices in Structural Models , 1990 .

[39]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[40]  Arnold Zellner,et al.  Introduction to Measurement with Theory , 2009 .

[41]  Greg L. Stewart,et al.  AN EXPLORATION OF MEMBER ROLES AS A MULTILEVEL LINKING MECHANISM FOR INDIVIDUAL TRAITS AND TEAM OUTCOMES , 2005 .

[42]  James C. Hayton,et al.  Factor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis , 2004 .

[43]  Robert E. Ployhart,et al.  An Evaluation of Generic Teamwork Skills Training with Action Teams: Effects on Cognitive and Skill-Based Outcomes , 2005 .

[44]  Neil Anderson,et al.  INTERNATIONAL VALIDITY GENERALIZATION OF GMA AND COGNITIVE ABILITIES: A EUROPEAN COMMUNITY META-ANALYSIS , 2003 .

[45]  D. Jackson,et al.  The six-factor model of personality: Facets from the Big Five. , 1996 .

[46]  Richard D. Arvey,et al.  MOTIVATIONAL COMPONENTS OF TEST TAKING , 1990 .

[47]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[48]  R. Goffin,et al.  Measuring job interview anxiety: Beyond weak knees and sweaty palms. , 2004 .

[49]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[50]  H. Williams.  Teams at work. , 2008 .

[51]  K. Murphy,et al.  Psychological Testing: Principles and Applications , 1988 .

[52]  R. P. McDonald,et al.  Choosing a multivariate model: Noncentrality and goodness of fit. , 1990 .

[53]  Kevin R. Murphy,et al.  Psychological testing: Principles and applications, 2nd ed. , 1991 .

[54]  B P O'Connor,et al.  SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[55]  Steven G. Rogelberg,et al.  Selection in Teams: An Exploration of the Teamwork Knowledge, Skills, and Ability Test , 2003 .

[56]  Frederick L. Oswald,et al.  On the robustness, bias, and stability of statistics from meta-analysis of correlation coefficients : Some initial Monte Carlo findings , 1998 .

[57]  Pilar Pérez-Santana,et al.  ‘Virtual Learning-by-Doing’ Teamwork KSA: Strategic Management Simulation as an Effective Tool , 2010 .

[58]  G. Graen.  Educating New Management Specialists From an Evidence- Based Perspective: A Proposal , 2009 .

[59]  N. Allen,et al.  Selection for Teams , 2005 .

[60]  Richard J. Klimoski,et al.  Training Undergraduates to Work in Organizational Teams , 2004 .

[61]  John R. Hollenbeck,et al.  Bridging the gap between I/O research and HR practice: Improving team composition, team training, and team task design , 2004 .

[62]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[63]  Douglas N. Jackson,et al.  Criterion validation of two approaches to performance appraisal: The behavioral observation scale and the relative percentile method , 1996 .

[64]  John E. Mathieu,et al.  A Temporally Based Framework and Taxonomy of Team Processes , 2001 .

[65]  D. Jackson,et al.  A five-factor versus six-factor model of personality structure , 1996 .

[66]  Ann Marie Ryan,et al.  Toward an Integrated Model of Applicant Faking Behavior , 2006 .

[67]  H. G. Osburn,et al.  Statistical power of tests of the situational specificity hypothesis in validity generalization studies: A cautionary note. , 1983 .

[68]  M. A. Campion,et al.  The Knowledge, Skill, and Ability Requirements for Teamwork: Implications for Human Resource Management , 1994 .

[69]  V. Bacharach,et al.  Psychometrics : An Introduction , 2007 .