The Integrated Workload Scale – Translation and validation of a subjective workload scale

The Integrated Workload Scale (IWS) is a scale for the assessment of subjective workload in real-time that was specifically designed for use with train traffic controllers. The IWS has nine anchor points that describe the degree of workload – ranging from ‘not demanding’ to ‘work too demanding’ – and captures the multi-dimensionality of workload by incorporating items that reflect time, demand and effort. In this paper, we describe how we translated the IWS into Dutch and subsequently validated the translation by having English-language students, Dutch students and Dutch train traffic controllers rate the individual items of the IWS according to the amount of workload each item conveyed, on a scale ranging from 0 (‘no workload at all’) to 150 (‘complete overload’). A comparison between the ratings of the English and Dutch items showed no significant differences, suggesting that the English and Dutch versions of the IWS are perceived similarly. Moreover, a comparison between the Dutch students and train traffic controllers also showed no apparent differences in the items' ratings. Regression analyses showed a strong linear component reflecting that the items were roughly equidistant on the rating scale. In a final analysis, alternate translations were tested as possible substitutes for some of the original items, but were not considered to substantially improve the scale. In sum, the similarity of the Dutch IWS to the original IWS, as well as the fact that train traffic controllers and students gave similar ratings on the Dutch IWS, suggest that it is a robust and reliable tool for subjective workload assessment across different populations and that it can be reliably translated to other languages.

[1]  Karel A. Brookhuis,et al.  On the assessment of (mental) workload and other subjective qualifications , 2002, Ergonomics.

[2]  F. Ruschitzka,et al.  Mental Stress Induces Prolonged Endothelial Dysfunction via Endothelin-A Receptors , 2002, Circulation.

[3]  Melcher Zeilstra,et al.  DEVELOPMENT AND IMPLEMENTATION OF A PREDICTIVE TOOL FOR OPTIMIZING WORKLOAD OF TRAIN DISPATCHERS , 2009 .

[4]  N. Taatgen,et al.  What happens when we switch tasks: pupil dilation in multitasking. , 2014, Journal of experimental psychology. Applied.

[5]  Mica R. Endsley,et al.  The Out-of-the-Loop Performance Problem and Level of Control in Automation , 1995, Hum. Factors.

[6]  Pamela S. Tsang,et al.  Mental Workload and Situation Awareness , 2006 .

[7]  Focus on psychometrics. Scaling stimuli with magnitude estimation. , 1992, Research in nursing & health.

[8]  J. C. Byers,et al.  Comparison of Four Subjective Workload Rating Scales , 1992 .

[9]  Lawrence L Ames,et al.  Revision and Verification of a Seven-Point Workload Estimate Scale , 1993 .

[10]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[11]  Ben Lewis-Evans,et al.  Self-report scales alone cannot capture mental workload , 2014, Cognition, Technology & Work.

[12]  Hankins Tc,et al.  A comparison of heart rate, eye activity, EEG and subjective measures of pilot mental workload during flight. , 1998, Aviation, space, and environmental medicine.

[13]  Sarah Sharples,et al.  Effects of level of signalling automation on workload and performance , 2012 .

[14]  Laura Pickup,et al.  The Integrated Workload Scale (IWS): a new self-report tool to assess railway signaller workload. , 2005, Applied ergonomics.

[15]  M Wilms,et al.  Subjective mental workload of Dutch train dispatchers: Validation of IWS in a practical setting , 2013 .

[16]  Gavriel Salvendy,et al.  Handbook of Human Factors and Ergonomics , 2005 .

[17]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[18]  Ari Widyanti,et al.  Adaptation of the Rating Scale Mental Effort (RSME) for use in Indonesia , 2013 .

[19]  E. Granholm,et al.  Pupillary responses index cognitive resource limitations. , 1996, Psychophysiology.

[20]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[21]  F.R.H. Zijlstra,et al.  Efficiency in work behaviour: A design approach for modern tools , 1993 .

[22]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .