Cognitive tasks and combined statistical methods to evaluate, model, and predict mental workload

Mental workload (MWL) is a concept that is used as a reference for assessing the mental cost of activities. In recent times, challenges related to user experience are determining the expected MWL value for a given activity and real-time adaptation of task complexity level to achieve or maintain desired MWL. As a consequence, it is important to have at least one task that can reliably predict the MWL level associated with a given complexity level. In this study, we used several cognitive tasks to meet this need, including the N-Back task, the commonly used reference test in the MWL literature, and the Corsi test. Tasks were adapted to generate different MWL classes measured via NASA-TLX and Workload Profile questionnaires. Our first objective was to identify which tasks had the most distinct MWL classes based on combined statistical methods. Our results indicated that the Corsi test satisfied our first objective, obtaining three distinct MWL classes associated with three complexity levels offering therefore a reliable model (about 80% accuracy) to predicted MWL classes. Our second objective was to achieve or maintain the desired MWL, which entailed the use of an algorithm to adapt the MWL class based on an accurate prediction model. This model needed to be based on an objective and real-time indicator of MWL. For this purpose, we identified different performance criteria for each task. The classification models obtained indicated that only the Corsi test would be a good candidate for this aim (more than 50% accuracy compared to a chance level of 33%) but performances were not sufficient to consider identifying and adapting the MWL class online with sufficient accuracy during a task. Thus, performance indicators require to be complemented by other types of measures like physiological ones. Our study also highlights the limitations of the N-back task in favor of the Corsi test which turned out to be the best candidate to model and predict the MWL among several cognitive tasks.

[1]  O. Pollatos,et al.  Ambient Light Conveying Reliability Improves Drivers' Takeover Performance without Increasing Mental Workload , 2022, Multimodal Technol. Interact..

[2]  C. Wickens,et al.  Human Mental Workload: A Survey and a Novel Inclusive Definition , 2022, Frontiers in Psychology.

[3]  Wenbin Li,et al.  Evaluating mental workload during multitasking in simulated flight , 2022, Brain and behavior.

[4]  L. Longo,et al.  An Evaluation of the EEG Alpha-to-Theta and Theta-to-Alpha Band Ratios as Indexes of Mental Workload , 2022, Frontiers in Neuroinformatics.

[5]  S. Poonguzhali,et al.  Machine learning-based approach for identifying mental workload of pilots , 2022, Biomed. Signal Process. Control..

[6]  Win-Ken Beh,et al.  MAUS: A Dataset for Mental Workload Assessmenton N-back Task Using Wearable Sensor , 2021, 2111.02561.

[7]  Xiaoke Chai,et al.  Evaluation of Mental Workload in Working Memory Tasks with Different Information Types Based on EEG , 2021, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[8]  Fabio Babiloni,et al.  A Review on Mental Stress Assessment Methods Using EEG Signals , 2021, Sensors.

[9]  K. Hong,et al.  Diagnosis of Mild Cognitive Impairment Using Cognitive Tasks: A Functional Near-Infrared Spectroscopy Study. , 2021, Current Alzheimer research.

[10]  R. Kazemi,et al.  Comparison of mental workload with N-Back test: A new design for NASA-task load index questionnaire , 2021 .

[11]  John Q. Young,et al.  Evidence for validity for the Cognitive Load Inventory for Handoffs , 2020, Medical education.

[12]  Nusrat Shaheen,et al.  A Novel Optimized Case-Based Reasoning Approach With K-Means Clustering and Genetic Algorithm for Predicting Multi-Class Workload Characterization in Autonomic Database and Data Warehouse System , 2020, IEEE Access.

[13]  F. Dehais,et al.  A Neuroergonomics Approach to Mental Workload, Engagement and Human Performance , 2020, Frontiers in Neuroscience.

[14]  Rifai Chai,et al.  The influence of mental fatigue on brain activity: Evidence from a systematic review with meta-analyses. , 2020, Psychophysiology.

[15]  A. Kramer,et al.  Physiological metrics of mental workload: A review of recent progress , 1990, Multiple-task performance.

[16]  Oceane Bel.  Dynamic performance enhancement of scientific networks and systems , 2020 .

[17]  Barry H. Kantowitz,et al.  Mental Workload , 2020, Encyclopedia of Behavioral Medicine.

[18]  O. Rosenblum,et al.  Non-binarité et transidentités à l’adolescence : une revue de la littérature , 2019, Neuropsychiatrie de l'Enfance et de l'Adolescence.

[19]  Junfeng Chen,et al.  Learning Spatial–Spectral–Temporal EEG Features With Recurrent 3D Convolutional Neural Networks for Cross-Task Mental Workload Assessment , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  Jim Nixon,et al.  Measuring mental workload using physiological measures: A systematic review. , 2019, Applied ergonomics.

[21]  Peter Chapman,et al.  Mental workload is reflected in driver behaviour, physiology, eye movements and prefrontal cortex activation. , 2018, Applied ergonomics.

[22]  Luca Longo,et al.  The Evolution of Cognitive Load Theory and the Measurement of Its Intrinsic, Extraneous and Germane Loads: A Review , 2018, H-WORKLOAD.

[23]  José J. Cañas,et al.  Latency Differences Between Mental Workload Measures in Detecting Workload Changes , 2018, H-WORKLOAD.

[24]  Anthony J. Ries,et al.  The Effect of Visual Task Difficulty on the Fixation-Related Lambda Response , 2018 .

[25]  Thea Radüntz,et al.  Dual Frequency Head Maps: A New Method for Indexing Mental Workload Continuously during Execution of Cognitive Tasks , 2017, Front. Physiol..

[26]  Luca Longo,et al.  Subjective Usability, Mental Workload Assessments and Their Impact on Objective Human Performance , 2017, INTERACT.

[27]  G. Riva,et al.  Brain-Computer Interface for Clinical Purposes: Cognitive Assessment and Rehabilitation , 2017, BioMed research international.

[28]  A. Bezerianos,et al.  Task-Independent Mental Workload Classification Based Upon Common Multiband EEG Cortical Connectivity , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Rosa H. M. Chan,et al.  An evaluation of mental workload with frontal EEG , 2017, PloS one.

[30]  Majid Fallahi,et al.  Effects of mental workload on physiological and subjective responses during traffic density monitoring: A field study. , 2016, Applied ergonomics.

[31]  Mahnaz Arvaneh,et al.  Filter bank common spatial patterns in mental workload estimation , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[32]  J. G. Hollands,et al.  Mental Workload, Stress, and Individual Differences: Cognitive and Neuroergonomic Perspectives , 2015 .

[33]  Jeffrey J. H. Cheung,et al.  Limitations of subjective cognitive load measures in simulation‐based procedural training , 2015, Medical education.

[34]  Shahrel Azmin Suandi,et al.  Hybrid Human Skin Detection Using Neural Network and K-Means Clustering Technique , 2015, Appl. Soft Comput..

[35]  Peter A Hancock,et al.  State of science: mental workload in ergonomics , 2015, Ergonomics.

[36]  Isabelle Milleville-Pennel,et al.  Do mental workload and presence experienced when driving a real car predispose drivers to simulator sickness? An exploratory study. , 2015, Accident; analysis and prevention.

[37]  Julie Paxion.  Complexité des situations, expérience, tension et vigilance : quels impacts sur la charge de travail et les performances de jeunes conducteurs ? , 2014 .

[38]  Claudine Mélan,et al.  A multidisciplinary approach of workload assessment in real-job situations: investigation in the field of aerospace activities , 2014, Front. Psychol..

[39]  T. Gog,et al.  Effects of pairs of problems and examples on task performance and different types of cognitive load , 2014 .

[40]  Caroline Martin.  La gestion de la charge mentale des contrôleurs aériens en-route : apports de l'eye-tracking dans le cadre du projet européen SESAR , 2013 .

[41]  Kevin Mandrick.  Application de la spectroscopie proche infrarouge dans la discrimination de la charge de travail. , 2013 .

[42]  Jinung An,et al.  Classification of frontal cortex haemodynamic responses during cognitive tasks using wavelet transforms and machine learning algorithms. , 2012, Medical engineering & physics.

[43]  Elizabeth W. Pang,et al.  Response inhibition in adults and teenagers: Spatiotemporal differences in the prefrontal cortex , 2012, Brain and Cognition.

[44]  L. Cuvelier.  Mesures quantitatives de la charge mentale : avancées, limites et usages pour la prévention des risques professionnels , 2012 .

[45]  C. Mélan,et al.  What is the relationship between mental workload factors and cognitive load types? , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[46]  J. Sweller.  Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load , 2010 .

[47]  J. Sweller,et al.  Cognitive load theory in health professional education: design principles and strategies , 2010, Medical education.

[48]  Richard D. Morey,et al.  Confidence Intervals from Normalized Data: A correction to Cousineau (2005) , 2008 .

[49]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[50]  B. Cain.  A Review of the Mental Workload Literature , 2007 .

[51]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[52]  E. Bornemann,et al.  Untersuchungen über den Grad der geistigen Beanspruchung , 1942, Arbeitsphysiologie.

[53]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[54]  André Dittmar,et al.  Mental workload in air traffic control: an index constructed from field tests. , 2004, Aviation, space, and environmental medicine.

[55]  Susana Rubio,et al.  Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA‐TLX, and Workload Profile Methods , 2004 .

[56]  É. Raufaste,et al.  Chapitre 8. Aspects intensifs de la cognition en situation de travail , 2004 .

[57]  E. Curran,et al.  Learning to control brain activity: A review of the production and control of EEG components for driving brain–computer interface (BCI) systems , 2003, Brain and Cognition.

[58]  R. Kessels,et al.  The Corsi Block-Tapping Task: Standardization and Normative Data , 2000, Applied neuropsychology.

[59]  M. J. Emerson,et al.  The Unity and Diversity of Executive Functions and Their Contributions to Complex “Frontal Lobe” Tasks: A Latent Variable Analysis , 2000, Cognitive Psychology.

[60]  ● Pytorch,et al.  Attention! , 1998, Trends in Cognitive Sciences.

[61]  P. Tsang,et al.  Diagnosticity and multidimensional subjective workload ratings. , 1996, Ergonomics.

[62]  Dick de Waard,et al.  The measurement of drivers' mental workload , 1996 .

[63]  C D Wickens,et al.  Assessment of pilot performance and mental workload in rotary wing aircraft. , 1993, Ergonomics.

[64]  F. T. Eggemeier,et al.  Recommendations for Mental Workload Measurement in a Test and Evaluation Environment , 1993 .

[65]  R. Heaton.  Wisconsin Card Sorting Test manual , 1993 .

[66]  Susan G. Hill,et al.  Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? In A , 1989 .

[67]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[68]  Thomas E. Nygren,et al.  The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload , 1988 .

[69]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[70]  Robert Karasek,et al.  Job decision latitude and mental strain: Implications for job redesign , 1979 .

[71]  H. Nelson.  A Modified Card Sorting Test Sensitive to Frontal Lobe Defects , 1976, Cortex.

[72]  Philip M. Corsi.  Human memory and the medial temporal region of the brain. , 1972 .

[73]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[74]  W. Kirchner.  Age differences in short-term retention of rapidly changing information. , 1958, Journal of experimental psychology.