Assessment of subjective workload in an anaesthesia simulator environment: reliability and validity

Background and objective For the subjective assessment of workload, Borg's Rating of Perceived Exertion (RPE) scale is a global measure of perceived workload during anaesthesia induction, maintenance and emergence in the real workplace. In the present study, validity and reliability of the RPE scale were analysed for a full-scale simulator environment using scenarios of induction of general anaesthesia with and without critical incidents. Methods Seventeen anaesthetists (professional experience 1–30 years) participated in this randomised cross-over trial. Each participant rated their workload using the RPE scale after three different simulator sessions. No critical incident was simulated in the 1st session. In a randomised order, workload was increased by simulation of a critical incident in the 2nd or 3rd session. For the analysis of validity and reliability, univariate and multivariate regression analysis and the concordance correlation coefficient were used. Results RPE scores were significantly increased after managing a simulated critical incident [13.0, 95% confidence interval (CI) 11.6–14.5] compared to normal anaesthesia induction (9.4, 95% CI 8.2–10.6; P < 0.001). Reliability was moderate (concordance correlation coefficient = 0.55; 95% CI 0.13–0.80) for uneventful sessions. Conclusion RPE scores were significantly increased after critical incidents during simulated anaesthesia induction and indicate good construct validity. Reliability may be impaired by the fact that the first session was announced to be without a critical incident. The RPE scale is easy to administer and a valid tool for subjective workload assessment in simulator settings. Reliability is moderate.

[1]  Andy Slovak,et al.  Introduction to Ergonomics , 2011 .

[2]  Matthew B Weinger,et al.  Multiple Measures of Anesthesia Workload During Teaching and Nonteaching Cases , 2004, Anesthesia and analgesia.

[3]  A. Byrne,et al.  Novel method of measuring the mental workload of anaesthetists during clinical practice. , 2009, British journal of anaesthesia.

[4]  Patrick W. Jordan,et al.  An Introduction to Usability , 1998 .

[5]  Chuan Zhou,et al.  Differences in Day and Night Shift Clinical Performance in Anesthesiology , 2008, Hum. Factors.

[6]  A F Smith,et al.  Methodological approaches to anaesthetists' workload in the operating theatre. , 2005, British journal of anaesthesia.

[7]  Stefan Kohlbecher,et al.  BASIC AND CLINICAL ASPECTS OF VERTIGO AND DIZZINESS EyeSeeCam: An Eye Movement–Driven Head Camera for the Examination of Natural Visual Exploration , 2009 .

[8]  G Schneider,et al.  Eye tracking for assessment of workload: a pilot study in an anaesthesia simulator environment. , 2011, British journal of anaesthesia.

[9]  D M Gaba,et al.  The Effect of Electronic Record Keeping and Transesophageal Echocardiography on Task Distribution, Workload, and Vigilance During Cardiac Anesthesia , 1997, Anesthesiology.

[10]  Tecnología do ar e espaço.  European Organisation for the Safety of Air Navigation , 2010 .

[11]  Matthew B Weinger,et al.  Assessment of the Intrarater and Interrater Reliability of an Established Clinical Task Analysis Methodology , 2002, Anesthesiology.

[12]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[13]  D M Gaba,et al.  Measuring the Workload of the Anesthesiologist , 1990, Anesthesia and analgesia.

[14]  Matthew B. Weinger,et al.  An Objective Methodology for Task Analysis and Workload Assessment in Anesthesia Providers , 1994, Anesthesiology.

[15]  G. Borg.  Psychophysical bases of perceived exertion. , 1982, Medicine and science in sports and exercise.

[16]  Matthew B. Weinger,et al.  Effects of Intraoperative Reading on Vigilance and Workload during Anesthesia Care in an Academic Medical Center , 2009, Anesthesiology.

[17]  C. Wickens.  Engineering psychology and human performance, 2nd ed. , 1992 .