Training Faculty in Nontechnical Skill Assessment: National Guidelines on Program Requirements

Objective: To develop guidelines for a faculty training program in nontechnical skill assessment in surgery. Background: Nontechnical skills in the operating room are critical for patient safety. The successful integration of these skills into workplace-based assessment is dependent upon the availability of faculty who are able to teach and assess them. At present, no guidelines exist regarding the training requirements for such faculty in surgical contexts. Methods: The development of the guidelines was carried out in several stages: stage 1—a detailed literature review on current training for nontechnical skill assessors; stage 2—semistructured interviews with a multidisciplinary panel (consisting of clinicians and psychologists/human factors specialists) of experts in surgical nontechnical skills; and stage 3—interview findings fed into an Expert Consensus Panel (ECP) Delphi approach to establish consensus regarding training requirements for faculty assessing nontechnical skills in surgery. Results: The ECP agreed that training in nontechnical skill assessment should be delivered by a multidisciplinary team consisting of clinicians and psychologists/human factors specialists. The ECP reached consensus regarding who should be targeted to be trained as faculty (including proficiency and revalidation requirements). Consensus was reached on 7 essential training program content elements (including training in providing feedback/debriefing) and 8 essential methods of evaluating the effectiveness of a “train-the-trainers” program. Conclusions: This study provides evidence-based guidelines that can be used to guide the development and evaluation of programs to educate faculty in the training and assessment of nontechnical skills. Uptake of these guidelines could accelerate the development of surgical expertise required for safe and high-quality patient care.

[1]  Nick Sevdalis,et al.  Observational teamwork assessment for surgery: content validation and tool refinement. , 2011, Journal of the American College of Surgeons.

[2]  M. Marshall Sampling for qualitative research. , 1996, Family practice.

[3]  E. Verdaasdonk,et al.  Objective assessment of technical surgical skills , 2010, The British journal of surgery.

[4]  A. K. Sachdeva,et al.  New directions in simulation-based surgical education and training: validation and transfer of surgical skills, use of nonsurgeons as faculty, use of simulation to screen and select surgery residents, and long-term follow-up of learners. , 2011, Surgery.

[5]  Rhona Flin,et al.  Evaluation of the Scrub Practitioners ’ List of Intraoperative Non-Technical Skills ( SPLINTS ) system , 2012 .

[6]  Michael Fordis,et al.  Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. , 2006, JAMA.

[7]  Rhona Flin,et al.  Experience matters: comparing novice and expert ratings of non‐technical skills using the NOTSS system , 2009, ANZ journal of surgery.

[8]  R Flin,et al.  Teaching surgeons about non-technical skills. , 2007, The surgeon : journal of the Royal Colleges of Surgeons of Edinburgh and Ireland.

[9]  Casey Mulqueen,et al.  Training Raters to Assess Resource Management Skills , 2001 .

[10]  Charles Vincent,et al.  Systems Approaches to Surgical Quality and Safety: From Concept to Measurement , 2004, Annals of surgery.

[11]  R. Key Dismukes,et al.  A Gold Standards Approach to Training Instructors to Evaluate Crew Performance , 2003 .

[12]  F. Hasson,et al.  A critical review of the Delphi technique as a research methodology for nursing. , 2001, International journal of nursing studies.

[13]  L. Feldman,et al.  Objective assessment, selection, and certification in surgery. , 2011, Surgical oncology.

[14]  P. Trucco,et al.  [Observational Team work Assessment for Surgery as Quality and Safety improvement tool]. , 2010, Igiene e sanita pubblica.

[15]  Eduardo Salas,et al.  Improving Teamwork in Organizations : Applications of Resource Management Training , 2001 .

[16]  Nick Sevdalis,et al.  Reliability of a revised NOTECHS scale for use in surgical teams. , 2008, American journal of surgery.

[17]  R. Reznick,et al.  Communication failures in the operating room: an observational classification of recurrent types and effects , 2004, Quality and Safety in Health Care.

[18]  Nick Sevdalis,et al.  Observational Teamwork Assessment for Surgery: Feasibility of Clinical and NonClinical Assessor Calibration With Short-Term Training , 2012, Annals of surgery.

[19]  K. T. Fong,et al.  Surgical team behaviors and patient outcomes. , 2009, American journal of surgery.

[20]  G. Hocking,et al.  Anaesthesia non-technical skills: Can anaesthetists be trained to reliably use this behavioural marker system in 1 day? , 2010, British journal of anaesthesia.

[21]  Paul Barach,et al.  A prospective observational study of human factors, adverse events, and patient outcomes in surgery for pediatric cardiac disease. , 2008, The Journal of thoracic and cardiovascular surgery.

[22]  R. Flin,et al.  Anaesthetists' Non-Technical Skills (ANTS): evaluation of a behavioural marker system. , 2003, British journal of anaesthesia.

[23]  R Flin,et al.  Development of a rating system for surgeons' non‐technical skills , 2006, Medical education.

[24]  G. Dunnington,et al.  The New ACS/APDS Skills Curriculum: Moving the Learning Curve Out of the Operating Room , 2008, Journal of Gastrointestinal Surgery.

[25]  Nick Sevdalis,et al.  The impact of nontechnical skills on technical performance in surgery: a systematic review. , 2012, Journal of the American College of Surgeons.

[26]  R. Brook,et al.  Consensus methods: characteristics and guidelines for use. , 1984, American journal of public health.

[27]  Rhona Flin,et al.  Surgeons’ Non-technical Skills in the Operating Room: Reliability Testing of the NOTSS Behavior Rating System , 2008, World Journal of Surgery.

[28]  Nick Sevdalis,et al.  Observational Teamwork Assessment for Surgery: Construct Validation With Expert Versus Novice Raters , 2009, Annals of surgery.

[29]  R. Flin,et al.  Patient safety in surgery: non-technical aspects of safe surgical performance , 2010, Patient safety in surgery.

[30]  P. McCulloch,et al.  The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre , 2009, Quality & Safety in Health Care.

[31]  Amitai Ziv,et al.  Simulation-Based Assessment and the Regulation of Healthcare Professionals , 2011, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[32]  G. Hanna,et al.  Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program , 2011, Surgical Endoscopy.

[33]  Nick Sevdalis,et al.  Self vs expert assessment of technical and non-technical skills in high fidelity simulation. , 2011, American journal of surgery.

[34]  G. Youngson Teaching and assessing non-technical skills. , 2011, The surgeon : journal of the Royal Colleges of Surgeons of Edinburgh and Ireland.