Crew Emergency Teamwork Assessment Measure (CETAM): A simulation-based first aid study on adapting and validating a tool for the assessment of nontechnical skills in airline crew

Introduction: Diverse tools dedicated to the assessment of various NTS components during cardiopulmonary resuscitation have been published for medical professionals, nevertheless there are no precise measures for the assessment of emergency resuscitation relevant in the context of non-medical community. The aim of the study is to combine and develop a valid and reliable nontechnical skills (NTS) tool for basic life support emergency resuscitation training via simulation learning for flight attendant. Methods: (1) Selection and combination of relevant items into a draft instrument with a UKM specialist team. (2) Obtaining the expert’s review from various specialty for content validity. (3) Instrument testing through pilot study on five recorded simulated case scenarios involving four different elements of nontechnical skills (teamwork, communication, decision making & situation awareness) which were demonstrated in a medical emergency (acute myocardial infarction) (4) Obtaining the internal consistency & inter-rater reliability of the instrument. Results: Through expert review, selected items had been found to have an excellent total content validity index of 1.00. A single mean of both raters yielded a good internal consistency of 0.77. Strong correlations of both ratings from raters in each video ranging from 0.69 to 0.93 (p < .05). There was a good inter-rater reliability (ICC 0.68) and a good agreement (Kappa 0.62 – 0.81) among raters. Conclusion: The CETAM was found to be a reliable and valid instrument and should be beneficial for the measurement of NTS, nevertheless further evaluation is required to fully determined its properties before reaching definitive conclusion.

[1]  Jeanne-Marie Guise,et al.  Validation of a Tool to Measure and Promote Clinical Teamwork , 2008, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[2]  M. Lynn.  Determination and quantification of content validity. , 1986, Nursing research.

[3]  A. Ziv,et al.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review , 2005, Medical teacher.

[4]  Cheryl Tatano Beck,et al.  The content validity index: are you sure you know what's being reported? Critique and recommendations. , 2006, Research in nursing & health.

[5]  T. Manser.  Teamwork and patient safety in dynamic domains of healthcare: a review of the literature , 2009, Acta anaesthesiologica Scandinavica.

[6]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[7]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .

[8]  S. New,et al.  Oxford NOTECHS II: A Modified Theatre Team Non-Technical Skills Scoring System , 2014, PloS one.

[9]  J. Fleiss.  Measuring nominal scale agreement among many raters. , 1971 .

[10]  L. Davis,et al.  Selection and use of content experts for instrument development. , 1997, Research in nursing & health.

[11]  S. Compton,et al.  Nursing research. Principles and methods: 7th edition , 2005 .

[12]  P. Baker,et al.  Non‐technical skills training to enhance patient safety: a systematic review , 2012, Medical education.

[13]  W. McGaghie,et al.  A critical review of simulation‐based medical education research: 2003–2009 , 2010, Medical education.

[14]  N Dowdall,et al.  “Is there a doctor on the aircraft?” Top 10 in-flight medical emergencies , 2000, BMJ : British Medical Journal.

[15]  Valéria Vernaschi Lima,et al.  Relato de experiência: o uso de simulações no processo de ensino-aprendizagem em medicina , 2009 .

[16]  Rhona Flin,et al.  Communication skills and error in the intensive care unit , 2007, Current opinion in critical care.

[17]  K. Hung,et al.  Medical volunteers in commercial flight medical diversions. , 2013, Aviation, space, and environmental medicine.

[18]  M. Brannick,et al.  Estimating the Reliability of Nontechnical Skills in Medical Teams. , 2017, Journal of surgical education.

[19]  Robyn Cant,et al.  Measuring non-technical skills in medical emergency care: a review of assessment measures , 2010, Open access emergency medicine : OAEM.

[20]  Amit Chandra,et al.  In-flight Medical Emergencies , 2013, The western journal of emergency medicine.

[21]  M. Odell.  Human factors and patient safety: changing roles in critical care. , 2011, Australian critical care : official journal of the Confederation of Australian Critical Care Nurses.

[22]  N. Sevdalis,et al.  Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR): Development and validation☆ , 2011, Resuscitation.

[23]  Peter Dieckmann,et al.  Development of instruments for assessment of individuals’ and teams’ non-technical skills in healthcare: a critical review , 2014, Cognition, Technology & Work.

[24]  James A Grand,et al.  Going DEEP: guidelines for building simulation-based team assessments , 2013, BMJ quality & safety.

[25]  L. Davis.  Instrument review: Getting the most from a panel of experts , 1992 .