A Measurement Tool for Simulation-Based Training in Emergency Medicine: The Simulation Module for Assessment of Resident Targeted Event Responses (SMARTER) Approach

The use of simulation in graduate medical education affords unique opportunities for increasing the quality of a resident’s educational experiences. Additionally, simulation poses a set of challenges that must be met to realize the full potential on learning and assessment practices. This article presents a methodology for creating simulation scenarios and accompanying measurement tools that are systematically linked in a direct and explicit manner to the Accreditation Council for Graduate Medical Education (ACGME) core competencies. This method, the Simulation Module for Assessment of Resident’s Targeted Event Responses (SMARTER), is an 8 step process that addresses the critical challenges of performance measurement in simulations and the need to document Accreditation Council for Graduate Medical Education core competency based learning outcomes. The SMARTER methodology is discussed in detail and 3 examples of scenario content and measurement tools generated with the SMARTER approach are provided. Additionally, results from an initial evaluation of the practicability and utility of the SMARTER measurement tools are discussed.

[1]  H. Breithaupt,et al.  Striking a balance , 2010, EMBO reports.

[2]  Karel Brookhuis,et al.  Handbook of Human Factors and Ergonomics Methods , 2009 .

[3]  James K Takayesu,et al.  A comprehensive medical simulation education curriculum for emergency medicine residents. , 2007, Annals of emergency medicine.

[4]  Rosemarie Fernandez,et al.  The use of simulation in emergency medicine: a research agenda. , 2007, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[5]  S. Dawson,et al.  Procedural simulation: a primer. , 2006, Journal of vascular and interventional radiology : JVIR.

[6]  Pascale Carayon,et al.  Handbook of human factors and ergonomics in health care and patient safety , 2006 .

[7]  Susan D. Swick,et al.  Assessing the ACGME Competencies in Psychiatry Training Programs , 2006, Academic psychiatry : the journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry.

[8]  J. McFetrich.  A structured literature review on the use of high fidelity patient simulators for teaching in emergency medicine , 2006, Emergency Medicine Journal.

[9]  Gary Klein,et al.  Working Minds: A Practitioner's Guide to Cognitive Task Analysis , 2006 .

[10]  John R Boulet,et al.  A Simulation-Based Acute Skills Performance Assessment for Anesthesia Training , 2005, Anesthesia and analgesia.

[11]  Eduardo Salas,et al.  Benefiting from mistakes: The impact of guided errors on learning, performance, and self-efficacy , 2005 .

[12]  A. Ziv,et al.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review , 2005, Medical teacher.

[13]  J. Woodhouse,et al.  Acute Care Skills in Anesthesia Practice: A Simulation-based Resident Performance Assessment , 2004, Anesthesiology.

[14]  D. Gaba.  The future vision of simulation in health care , 2004, Quality and Safety in Health Care.

[15]  Jennifer E. Fowlkes,et al.  Targeted Acceptable Responses to Generated Events or Tasks (TARGETs) , 2004 .

[16]  S. Swing,et al.  Integrating the accreditation council for graduate medical education core competencies into the model of the clinical practice of emergency medicine. , 2004, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[17]  David Murray,et al.  Reliability and Validity of a Simulation-based Acute Care Skills Assessment for Medical Students and Residents , 2003, Anesthesiology.

[18]  A. Ziv,et al.  Simulation-based medical education: an ethical imperative. , 2006, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[19]  S. Swing,et al.  Assessing the ACGME general competencies: general considerations and assessment methods. , 2002, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[20]  David Murray,et al.  An acute care skills evaluation for graduating medical students: a pilot study using clinical simulation , 2002, Medical education.

[21]  Shawna J Perry,et al.  Human factors and ergonomics in the emergency department. , 2002, Annals of emergency medicine.

[22]  Jeffrey T. Hansberger,et al.  Improving Rater Calibration in Aviation: A Case Study , 2002 .

[23]  Florian Jentsch,et al.  Generating Line-Oriented Flight Simulation Scenarios with the Rrloe Computerized Tool Set , 2001 .

[24]  Ling Rothrock,et al.  Using Time Windows to Evaluate Operator Performance , 2001 .

[25]  Irwin L. Goldstein,et al.  Training in Organizations , 2001 .

[26]  Valerie L. Shalin,et al.  Cognitive task analysis , 2000 .

[27]  Eduardo Salas,et al.  Training team problem solving skills: an event-based approach ? ? The views herein are those of the , 1999 .

[28]  Jennifer E. Fowlkes,et al.  Performance Measurement in Distributed Environments: Initial Results and Implications for Training , 1999 .

[29]  Jennifer E. Fowlkes,et al.  Event-based approach to training (EBAT). , 1998 .

[30]  Florian Jentsch,et al.  Rapidly Reconfigurable Event-Set Based Line Operational Evaluation Scenarios , 1997 .

[31]  Eduardo Salas,et al.  Team Performance Assessment and Measurement: Theory, Methods, and Applications. Series in Applied Psychology. , 1997 .

[32]  E. Salas,et al.  Enhancing teamwork in complex environments through team training. , 1997, Group dynamics : theory, research, and practice : the official journal of Division 49, Group Psychology and Group Psychotherapy of the American Psychological Association.

[33]  Janis A. Cannon-Bowers,et al.  Performance measurement tools for enhancing team decision-making training. , 1997 .

[34]  R. Bakeman,et al.  Observing interaction: An introduction to sequential analysis, 2nd ed. , 1997 .

[35]  Norman E. Lane,et al.  Improving the Measurement of Team Performance: The TARGETs Methodology , 1994 .

[36]  M. Smith,et al.  Training in Organizations , 1991 .

[37]  Robert T. Hays,et al.  Simulation Fidelity in Training System Design , 1989 .

[38]  Roger Bakeman,et al.  Observing Interaction: An Introduction to Sequential Analysis , 1986 .

[39]  Donald Vreuls,et al.  Human-System Performance Measurement in Training Simulators , 1985 .

[40]  Walter Schneider,et al.  Training High-Performance Skills: Fallacies and Guidelines , 1985 .

[41]  J. Ende.  Feedback in clinical medical education. , 1983, JAMA.