Objective Structured Assessment of Debriefing: Bringing Science to the Art of Debriefing in Surgery

Objective:To identify the features of effective debriefing and to use this to develop and validate a tool for assessing such debriefings. Introduction:Simulation-based training has become an accepted means of surgical skill acquisition. A key component of this is debriefing—yet there is a paucity of research to guide best practice. Methods:Phase 1—Identification of best practice and tool development. A search of the Medline, Embase, PsycINFO, and ERIC databases identified current evidence on debriefing. End-user input was obtained through 33 semistructured interviews conducted with surgeons (n = 18) and other operating room personnel (n = 15) from 3 continents (UK, USA, Australia) using standardized qualitative methodology. An expert panel (n = 7) combined the data to create the Objective Structured Assessment of Debriefing (OSAD) tool. Phase 2—Psychometric testing. OSAD was tested for feasibility, reliability, and validity by 2 independent assessors who rated 20 debriefings following high-fidelity simulations. Results:Phase 1: 28 reports on debriefing were retrieved from the literature. Key components of an effective debriefing identified from these reports and the 33 interviews included: approach to debriefing, learning environment, learner engagement, reaction, reflection, analysis, diagnosis of strengths and areas for improvement, and application to clinical practice. Phase 2: OSAD was feasible, reliable [inter-rater ICC (intraclass correlation coefficient) = 0.88, test–retest ICC = 0.90], and face and content valid (content validity index = 0.94). Conclusions:OSAD provides an evidence-based, end-user informed approach to debriefing in surgery. By quantifying the quality of a debriefing, OSAD has the potential to identify areas for improving practice and to optimize learning during simulation-based training.

[1]  David Smith,et al.  Debriefing in Experience-Based Learning. , 1986 .

[2]  A. Ziv,et al.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review , 2005, Medical teacher.

[3]  R. J. Bogumil,et al.  The reflective practitioner: How professionals think in action , 1985, Proceedings of the IEEE.

[4]  R. Satava,et al.  Virtual Reality Training Improves Operating Room Performance: Results of a Randomized, Double-Blinded Study , 2002, Annals of surgery.

[5]  W. Baine,et al.  The Agency for Healthcare Research and Quality , 2006, Italian Journal of Public Health.

[6]  Akihito Kamata,et al.  Developing and Validating Rapid Assessment Instruments , 2009 .

[7]  Harry Owen,et al.  GREAT simulation debriefing , 2006, Medical education.

[8]  Marcus Rall,et al.  Key elements of debriefing for simulator training , 2000 .

[9]  Eduardo Salas,et al.  Debriefing medical teams: 12 evidence-based best practices and tips. , 2008, Joint Commission journal on quality and patient safety.

[10]  Roger Chow,et al.  Value of Debriefing during Simulated Crisis Management: Oral versus Video-assisted Oral Feedback , 2006, Anesthesiology.

[11]  J. Rudolph,et al.  Debriefing with Good Judgment , 2013 .

[12]  Lisa L. Schlitzkus,et al.  Is your residency program ready for Generation Y? , 2010, Journal of surgical education.

[13]  David M. Gaba,et al.  The Role of Debriefing in Simulation-Based Learning , 2007, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[14]  P. Croskerry,et al.  Cognitive versus technical debriefing after simulation training. , 2006, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[15]  Kathleen A. Johnson,et al.  Support for Simulation-based Surgical Education through American College of Surgeons – Accredited Education Institutes , 2008, World Journal of Surgery.

[16]  Cheryl Tatano Beck,et al.  The content validity index: are you sure you know what's being reported? Critique and recommendations. , 2006, Research in nursing & health.

[17]  Nick Sevdalis,et al.  Self vs expert assessment of technical and non-technical skills in high fidelity simulation. , 2011, American journal of surgery.

[18]  H. DeVon,et al.  A psychometric toolbox for testing validity and reliability. , 2007, Journal of nursing scholarship : an official publication of Sigma Theta Tau International Honor Society of Nursing.

[19]  V. LeBlanc,et al.  Efficacy of high-fidelity simulation debriefing on the performance of practicing anaesthetists in simulated scenarios. , 2009, British journal of anaesthesia.

[20]  Robert Simon,et al.  Debriefing as formative assessment: closing performance gaps in medical education. , 2008, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[21]  D. Gaba.  The future vision of simulation in health care , 2004, Quality and Safety in Health Care.

[22]  Daniel B. Jones,et al.  Surgical skills training and simulation. , 2009, Current problems in surgery.

[23]  William C McGaghie,et al.  Effect of practice on standardised learning outcomes in simulation‐based medical education , 2006, Medical education.

[24]  Kristina Thomas Dreifuerst,et al.  THE ESSENTIALS of DEBRIEFING in Simulation Learning: A Concept Analysis , 2009, Nursing education perspectives.

[25]  Nick Sevdalis,et al.  Factors compromising safety in surgery: stressful events in the operating room. , 2010, American journal of surgery.

[26]  Roger Kneebone,et al.  Simulation, safety and surgery , 2010, Quality and Safety in Health Care.

[27]  Peter Dieckmann,et al.  The art and science of debriefing in simulation: Ideal and practice , 2009, Medical teacher.

[28]  Rajesh Aggarwal,et al.  Implementation of Virtual Online Patient Simulation , 2011, MMVR.

[29]  R. Kneebone.  Simulation in surgical training: educational issues and practical implications , 2003, Medical education.

[30]  Peter E. Rivard,et al.  Debriefing with good judgment: combining rigorous feedback with genuine inquiry. , 2007, Anesthesiology clinics.

[31]  Nick Sevdalis,et al.  Identifying best practice guidelines for debriefing in surgery: a tri-continental study. , 2012, American journal of surgery.

[32]  N Mays,et al.  Qualitative Research: Rigour and qualitative research , 1995 .

[33]  David M Gaba,et al.  So many roads: facilitated debriefing in healthcare. , 2006, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[34]  H. Owen,et al.  Learning on a simulator does transfer to clinical practice. , 2009, Resuscitation.

[35]  L. Lederman.  Debriefing: Toward a Systematic Assessment of Theory and Practice , 1992 .

[36]  Daniel B. Raemer,et al.  There's No Such Thing as “Nonjudgmental” Debriefing: A Theory and Method for Debriefing with Good Judgment , 2006, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[37]  James M McGreevy,et al.  Briefing and debriefing in the operating room using fighter pilot crew resource management. , 2007, Journal of the American College of Surgeons.

[38]  Barbara Steinwachs.  How to Facilitate a Debriefing , 1992 .

[39]  Linda C. Lederman,et al.  Debriefing: A Critical Reexamination of the Postexperience Analytic Process with Implications for Its Effective Use. , 1984 .

[40]  J. Paige,et al.  Examining the Effectiveness of Debriefing at the Point of Care in Simulation-Based Operating Room Team Training , 2008 .

[41]  G. Moneta,et al.  Evaluation of distributed practice schedules on retention of a newly acquired surgical skill: a randomized trial. , 2011, American journal of surgery.

[42]  G. Hamad,et al.  Postoperative video debriefing reduces technical errors in laparoscopic surgery. , 2007, American journal of surgery.

[43]  Benjamin S Abella,et al.  Improving in-hospital cardiac arrest process and outcomes with performance debriefing. , 2008, Archives of internal medicine.

[44]  W. McGaghie,et al.  A critical review of simulation‐based medical education research: 2003–2009 , 2010, Medical education.

[45]  K Henriksen,et al.  Understanding Quality and Safety Problems in the Ambulatory Environment: Seeking Improvement With Promising Teamwork Tools and Strategies -- Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 3: Performance and Tools) , 2008 .

[46]  J. M. Davies,et al.  Qualitative Research in Health Care , 1996, Journal of the Royal College of Physicians of London.

[47]  Nick Sevdalis,et al.  Mental Practice Enhances Surgical Technical Skills: A Randomized Controlled Study , 2011, Annals of surgery.

[48]  Nicole K Roberts,et al.  The briefing, intraoperative teaching, debriefing model for teaching in the operating room. , 2009, Journal of the American College of Surgeons.

[49]  G. Dunnington,et al.  The New ACS/APDS Skills Curriculum: Moving the Learning Curve Out of the Operating Room , 2008, Journal of Gastrointestinal Surgery.

[50]  Donald A. Sch.  The reflective practitioner: how professionals think in action , 1983 .

[51]  Vicki R LeBlanc,et al.  Personalized Oral Debriefing Versus Standardized Multimedia Instruction After Patient Crisis Simulation , 2009, Anesthesia and analgesia.

[52]  Inuin M. Rubin,et al.  The ABCs of Effective Feedback: A Guide for Caring Professionals , 1997 .

[53]  W. McGaghie,et al.  What is feedback in clinical education? , 2008, Medical education.

[54]  N. Seymour.  VR to OR: A Review of the Evidence that Virtual Reality Simulation Improves Operating Room Performance , 2008, World Journal of Surgery.