Tools for evaluating team performance in simulation-based training

Teamwork training constitutes one of the core approaches for moving healthcare systems toward increased levels of quality and safety, and simulation provides a powerful method of delivering this training, especially for face-paced and dynamic specialty areas such as Emergency Medicine. Team performance measurement and evaluation plays an integral role in ensuring that simulation-based training for teams (SBTT) is systematic and effective. However, this component of SBTT systems is overlooked frequently. This article addresses this gap by providing a review and practical introduction to the process of developing and implementing evaluation systems in SBTT. First, an overview of team performance evaluation is provided. Second, best practices for measuring team performance in simulation are reviewed. Third, some of the prominent measurement tools in the literature are summarized and discussed relative to the best practices. Subsequently, implications of the review are discussed for the practice of training teamwork in Emergency Medicine.

[1]  Irwin L. Goldstein,et al.  Training in organizations: Needs assessment, development, and evaluation , 1986 .

[2]  George C. Thornton,et al.  Assessment Centers in Human Resource Management , 1992 .

[3]  Norman E. Lane,et al.  Improving the Measurement of Team Performance: The TARGETs Methodology , 1994 .

[4]  N. Black Why we need observational studies to evaluate the effectiveness of health care , 1996, BMJ.

[5]  D. P. Baker,et al.  Analyzing team performance: in the eye of the beholder? , 1996, Military psychology : the official journal of the Division of Military Psychology, American Psychological Association.

[6]  Winston Bennett,et al.  A META-ANALYSIS OF THE RELATIONS AMONG TRAINING CRITERIA , 1997 .

[7]  E. Salas,et al.  A Framework for Developing Team Performance Measures in Training , 1997 .

[8]  L. L. Cummings,et al.  A test of the influence of goal orientation on the feedback-seeking process. , 1997, The Journal of applied psychology.

[9]  Jennifer E. Fowlkes,et al.  Event-based approach to training (EBAT). , 1998 .

[10]  R. Key Dismukes,et al.  Facilitation and Debriefing in Aviation Training and Operations , 2000 .

[11]  Rhona Flin,et al.  Developing a Method for Evaluating Crew Resource Management Skills: A European Perspective , 2002 .

[12]  George C. Thornton,et al.  Developing Organizational Simulations : A Guide for Practitioners and Students , 2003 .

[13]  J Harvey Magee Validation of medical modeling & simulation training devices and systems. , 2003, Studies in health technology and informatics.

[14]  Rhona Flin,et al.  Rating non-technical skills: developing a behavioural marker system for use in anaesthesia , 2004, Cognition, Technology & Work.

[15]  S Undre,et al.  Developing observational measures of performance in surgical teams , 2004, Quality and Safety in Health Care.

[16]  J. Sexton,et al.  Translating teamwork behaviours from aviation to healthcare: development of behavioural markers for neonatal resuscitation , 2004, Quality and Safety in Health Care.

[17]  Dana E. Sims,et al.  Is there a “Big Five” in Teamwork? , 2005 .

[18]  Eduardo Salas,et al.  Beyond facilitation: an improved CRM debrief for safety training , 2005 .

[19]  Susan D Horn,et al.  Performance measures and clinical outcomes. , 2006, JAMA.

[20]  E. Salas,et al.  Markers for enhancing team cognition in complex environments: the power of team performance diagnosis. , 2007, Aviation, space, and environmental medicine.

[21]  Douglas A Wiegmann,et al.  The Mayo High Performance Teamwork Scale: Reliability and Validity for Evaluating Key Crew Resource Management Skills , 2007, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[22]  Roxane Gardner,et al.  Using the Communication and Teamwork Skills (CATS) Assessment to measure health care team performance. , 2007, Joint Commission journal on quality and patient safety.

[23]  Eduardo Salas,et al.  Promoting teamwork: an event-based approach to simulation-based teamwork training for emergency medicine residents. , 2008, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[24]  Eduardo Salas,et al.  The wisdom of collectives in organizations: An update of the teamwork competencies. , 2008 .

[25]  Tammy L. Rapp,et al.  Team Effectiveness 1997-2007: A Review of Recent Advancements and a Glimpse Into the Future , 2008 .

[26]  Eduardo Salas,et al.  Communicating, coordinating, and cooperating when lives depend on it: tips for teamwork. , 2008, Joint Commission journal on quality and patient safety.

[27]  J. Mathieu,et al.  Goal orientation dispositions and performance trajectories: The roles of supplementary and complementary situational inducements. , 2008 .

[28]  Eduardo Salas,et al.  Measuring team performance in simulation-based training: adopting best practices for healthcare. , 2008, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[29]  E. Salas,et al.  Building team and technical competency for obstetric emergencies: the mobile obstetric emergencies simulator (MOES) system. , 2009, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[30]  E. Petrusa,et al.  Assessing teamwork in medical education and practice: Relating behavioural teamwork ratings and clinical performance , 2009, Medical teacher.