Crew Resource Management Training Effectiveness: A Meta-Analysis and Some Critical Needs

Empirical studies of crew resource management (CRM) training effectiveness were subjected to meta-analysis. Sixteen CRM evaluation studies were found to fulfill the a priori criteria for inclusion in the meta-analysis. The metrics of CRM training effectiveness analyzed were reactions, attitudes, knowledge, and behaviors. CRM-trained participants responded positively to CRM (a mean of 4 on a 5—point Likert scale). The training had large effects on the participants' attitudes and behaviors and a medium effect on their knowledge. The findings from the meta-analysis are encouraging for the effectiveness of CRM training. However, there is a need for researchers and reviewers to be more rigorous about the data included in research reporting CRM evaluation to allow effect sizes to be calculated.

[1]  Clint A. Bowers,et al.  The Design and Delivery of Crew Resource Management Training: Exploiting Available Resources , 2000, Hum. Factors.

[2]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[3]  Paul O'Connor,et al.  TECHNIQUES USED TO EVALUATE CREW RESOURCE MANAGEMENT TRAINING: A LITERATURE REVIEW , 2002 .

[4]  J. Henry,et al.  A meta-analytic review of verbal fluency deficits in Huntington's disease. , 2005, Neuropsychology.

[5]  F. Schmidt.  Meta-Analysis , 2008 .

[6]  W. Shadish,et al.  How interchangeable are different estimators of effect size? , 1996, Journal of consulting and clinical psychology.

[7]  E. Salas,et al.  Methods, tools, and strategies for team training. , 1997 .

[8]  L. Sjodahl.  Evaluation of Training. , 1975 .

[9]  Eduardo Salas,et al.  A Comparison of Two Types of Training Interventions on Team Communication Performance , 1990 .

[10]  Cidambi Srinivasan,et al.  Combining Information: Statistical Issues and Opportunities for Research , 1993 .

[11]  Susan E. Jackson,et al.  Meta-analysis analysis. , 1987 .

[12]  K. Fish,et al.  Anaesthesia crisis resource management training: an intimidating concept, a rewarding experience , 1996, Canadian journal of anaesthesia = Journal canadien d'anesthesie.

[13]  Mercer Jennifer Ann,et al.  PUBLICATION manual of the American Psychological Association. , 1952, Psychological bulletin.

[14]  Captain H. C. Alger.  Cockpit resource management. , 1989, Aviation, space, and environmental medicine.

[15]  Brian Mullen,et al.  Advanced BASIC meta-analysis , 1989 .

[16]  R L Helmreich,et al.  The evolution of Crew Resource Management training in commercial aviation. , 1999, The International journal of aviation psychology.

[17]  Katherine A. Wilson,et al.  A Checklist for Crew Resource Management Training , 2006 .

[18]  N. Crepaz,et al.  Quality of study methods in individual- and group-level HIV intervention research: critical reporting elements. , 2004, AIDS education and prevention : official publication of the International Society for AIDS Education.

[19]  John C. Morey,et al.  Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum? , 2004, Quality and Safety in Health Care.

[20]  Winston Bennett,et al.  Effectiveness of training in organizations: a meta-analysis of design and evaluation features. , 2003, The Journal of applied psychology.

[21]  D. P. Gaver,et al.  Combining Information: Statistical Issues and Opportunities for Research , 1993 .

[22]  Richard A. Guzzo,et al.  The Effects of Psychologically Based Intervention Programs on Worker Productivity: A Meta-Analysis. , 1985 .

[23]  John C. Morey,et al.  Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. , 2002, Health services research.

[24]  R L Helmreich,et al.  Cockpit Management Attitudes , 1984, Human factors.

[25]  Steven E. Gregorich,et al.  CREW RESOURCE MANAGEMENT TRAINING ASSESSMENT. , 1993 .

[26]  Clint A. Bowers,et al.  Team Training in the Skies: Does Crew Resource Management (CRM) Training Work? , 2001, Hum. Factors.

[27]  Ellen M. Whitener,et al.  Confusion of confidence intervals and credibility intervals in meta-analysis. , 1990 .

[28]  Cathy C. Clothier.  Behavioral interactions across various aircraft types - Results of systematic observations of line operations and simulations , 1991 .

[29]  Martha Reeves,et al.  Evaluation of Training , 1993 .

[30]  A. C. Hamblin,et al.  Evaluation and control of training , 1974 .

[31]  E Salas,et al.  The role of trainee knowledge structures in aviation team environments. , 1997, The International journal of aviation psychology.

[32]  S D Small,et al.  Anesthesia crisis resource management: real-life simulation training in operating room crises. , 1995, Journal of clinical anesthesia.

[33]  Lynne Martin,et al.  Behavioral Markers for Crew Resource Management: A Review of Current Practice , 2001 .

[34]  E. Salas,et al.  The science of training: a decade of progress. , 2001, Annual review of psychology.

[35]  Clint A. Bowers,et al.  A Methodology for Enhancing Crew Resource Management Training , 1999, Hum. Factors.

[36]  Rhona Flin,et al.  Crew resource management: improving team work in high reliability industries , 2002 .

[37]  Richard P. DeShon,et al.  Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. , 2002, Psychological methods.

[38]  G. Alliger,et al.  KIRKPATRICK'S LEVELS OF TRAINING CRITERIA: THIRTY YEARS LATER , 1989 .

[39]  Rhona Flin,et al.  Crew Resource Management training for offshore oil production teams , 2003 .

[40]  J Fisher,et al.  Does crew resource management training work? , 2000, Air medical journal.

[41]  Eduardo Salas,et al.  Does CRM Training Improve Teamwork Skills in the Cockpit?: Two Evaluation Studies , 1999, Hum. Factors.

[42]  Dennis K. Leedom,et al.  Improving Team Coordination: A Case for Behavior-Based Training , 1995 .

[43]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[44]  S. Gregorich,et al.  The structure of cockpit management attitudes. , 1990, The Journal of applied psychology.

[45]  Eduardo Salas,et al.  Aviation Computer Games for Crew Resource Management Training , 1993 .

[46]  Timothy T. Baldwin,et al.  TRANSFER OF TRAINING: A REVIEW AND DIRECTIONS FOR FUTURE RESEARCH , 1988 .

[47]  Eduardo Salas,et al.  Does Crew Resource Management Training Work? An Update, an Extension, and Some Critical Needs , 2006, Hum. Factors.

[48]  S. Thompson.  Social Learning Theory , 2008 .

[49]  Paul O'Connor.  Methods used to evaluate the effectiveness of CRM training: A literature review. , 2002 .

[50]  Katherine A. Wilson,et al.  Crew Resource Management Training Research, Practice, and Lessons Learned: , 2006 .

[51]  E. Salas,et al.  Enhancing teamwork in complex environments through team training. , 1997, Group dynamics : theory, research, and practice : the official journal of Division 49, Group Psychology and Group Psychotherapy of the American Psychological Association.