Measuring mental workload with the NASA-TLX needs to examine each dimension rather than relying on the global score: an example with driving

Abstract The distinction between several components of mental workload is often made in the ergonomics literature. However, measurements used are often established from a global score, notably with several questionnaires that originally reflect several dimensions. The present study tested the effect of driving situation complexity, experience and subjective levels of tension and alertness on each dimension of the NASA-TLX questionnaire of workload, in order to highlight the potential influence of intrinsic, extraneous and germane load factors. The results showed that, in complex situation, mental, temporal and physical demand (load dimensions) increased, and that novice drivers presented high physical demand when subjective tension was low on performance. Moreover, increase of mental and physical demand increased effort. It thus, appears essential to distinguish the different components of mental workload used in the NASA-TLX questionnaire. Practitioner Summary: Currently, global score of NASA-TLX questionnaire is used to measure mental workload. Here, we considered independently each dimension of NASA-TLX, and results showed that mental load factors (driving situation complexity, experience, subjective tension and alertness) had a different effect on dimensions, questioning global score use to evaluate workload.

[1]  T. Gog,et al.  Uncovering Expertise-Related Differences in Troubleshooting Performance: Combining Eye Movement and Concurrent Verbal Protocol Data , 2005 .

[2]  Mark Bullen,et al.  What is Instructional Design , 2014 .

[3]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[4]  C. Guilleminault,et al.  Fatigue, sleep restriction and driving performance. , 2005, Accident; analysis and prevention.

[5]  Aline Chevalier,et al.  The use of Tholos software for combining measures of mental workload: Toward theoretical and methodological improvements , 2008, Behavior research methods.

[6]  Edith Galy,et al.  Mnemonic Processing in Air Traffic Controllers: Effects of Task Parameters and Work Organization , 2007 .

[7]  H. Hadj-Mabrouk,et al.  Chronobiologie de la vigilance Approche d'application dans le domaine de la sécurité routière , 2001 .

[8]  Ravi Paul,et al.  Analyzing the structure of expert knowledge , 2006, Inf. Manag..

[9]  Thomas E. Nygren,et al.  The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload , 1988 .

[10]  W. Schnotz,et al.  A Reconsideration of Cognitive Load Theory , 2007 .

[11]  Dick de Waard,et al.  The measurement of drivers' mental workload , 1996 .

[12]  C. Collet,et al.  Stroop color–word test, arousal, electrodermal activity and performance in a critical driving situation , 2005, Biological Psychology.

[13]  Robert M Wachter,et al.  Advancing the next generation of handover research and practice with cognitive load theory , 2016, BMJ Quality & Safety.

[14]  A W Gaillard,et al.  Comparing the concepts of mental load and stress. , 1993, Ergonomics.

[15]  Saskia de Craen,et al.  The development of a method to measure speed adaptation to traffic complexity: identifying novice, unsafe, and overconfident drivers. , 2008, Accident; analysis and prevention.

[16]  Robert E. Thayer,et al.  Problem perception, optimism, and related states as a function of time of day (diurnal rhythm) and moderate exercise: Two arousal systems in interaction , 1987 .

[17]  Stephane Espie,et al.  Driving Simulators Validation: The Issue of Transferability of Results Acquired on Simulator , 2005 .

[18]  Shin-Yuan Hung.  Expert versus novice use of the executive support systems: an empirical study , 2003, Inf. Manag..

[19]  Iris Vessey,et al.  The effect of information presentation on decision making: A cost-benefit analysis , 1994, Inf. Manag..

[20]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[21]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[22]  Peter A Hancock,et al.  State of science: mental workload in ergonomics , 2015, Ergonomics.

[23]  Lewis A. Bizo,et al.  Evaluating the effects of team-skills training on subjective workload , 2011 .

[24]  Jing Xu,et al.  Quantitative assessment of workload and stressors in clinical radiation oncology. , 2012, International journal of radiation oncology, biology, physics.

[25]  Wolfgang Fastenmeier,et al.  DIE VERKEHRSSITUATION ALS ANALYSEEINHEIT IM VERKEHRSSYSTEM , 1995 .

[26]  Isabelle Milleville-Pennel,et al.  Remote collaboration in virtual reality: asymmetrical effects of task distribution on spatial processing and mental workload , 2016, Virtual Reality.

[27]  Erin V. Lehman,et al.  Development of an Instrument , 2005 .

[28]  C. Mélan,et al.  What is the relationship between mental workload factors and cognitive load types? , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[29]  Claudine Mélan,et al.  A multidisciplinary approach of workload assessment in real-job situations: investigation in the field of aerospace activities , 2014, Front. Psychol..

[30]  Dylan M. Jones,et al.  Refining the measurement of mood: The UWIST Mood Adjective Checklist , 1990 .

[31]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[32]  J. Sweller.  COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[33]  David Meister,et al.  Behavioral foundations of system development , 1984 .

[34]  Fu Shan,et al.  Using NASA-TLX to evaluate the flight deck design in Design Phase of Aircraft , 2011 .

[35]  A. Muzet,et al.  Correlation between driving errors and vigilance level: influence of the driver's age , 2004, Physiology & Behavior.

[36]  Edith Galy,et al.  Effects of Cognitive Appraisal and Mental Workload Factors on Performance in an Arithmetic Task , 2015, Applied psychophysiology and biofeedback.

[37]  Catherine Berthelon,et al.  The Evaluation of Traditional and Early Driver Training With Simulated Accident Scenarios , 2011, Hum. Factors.

[38]  Catherine Berthelon,et al.  Overload depending on driving experience and situation complexity: Which strategies faced with a pedestrian crossing? , 2015, Applied ergonomics.

[39]  S. Miyake.  Multivariate workload evaluation combining physiological and subjective measures. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[40]  W B Verwey,et al.  On-line driver workload estimation. Effects of road situation and age on secondary task measures , 2000, Ergonomics.

[41]  B. Radošević-Vidaček,et al.  Circadian Components in Energy and Tension and Their Relation to Physiological Activation and Performance , 2004, Chronobiology international.

[42]  Nicolas Debue,et al.  What does germane load mean? An empirical contribution to the cognitive load theory , 2014, Front. Psychol..

[43]  Jean-Michel Hoc,et al.  Cognitive styles as an explanation of experts' individual differences: A case study in computer-assisted troubleshooting diagnosis , 2006, Int. J. Hum. Comput. Stud..

[44]  T. Gog,et al.  Development of an instrument for measuring different types of cognitive load , 2013, Behavior Research Methods.

[45]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[46]  Barry H. Kantowitz,et al.  Mental Workload , 2020, Encyclopedia of Behavioral Medicine.

[47]  Li Yang,et al.  Research on Layered Resource Discovery Model in Grid , 2011 .

[48]  C. Colleta,et al.  Autonomic nervous system and subjective ratings of strain in air-traffic control , 2008 .

[49]  R. Thayer.  Activation-Deactivation Adjective Check List: Current Overview and Structural Analysis , 1986 .

[50]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .