Mental Workload Monitoring: New Perspectives from Neuroscience

Mental Workload is nowadays a keyword used and sometimes abused in life sciences. The present chapter aims at introducing the concept of mental workload, its relevance for Human Factor research and the current needs of applied disciplines in a clear and effective way. This paper will present a state-of-art overview of recent outcomes produced by neuroscientific research to highlight current trends in this field. The present paper will offer an overview of and some examples of what neuroscience has to offer to mental workload-related research.

[1]  Thibault Gateau,et al.  Auditory Alarm Misperception in the Cockpit: An EEG Study of Inattentional Deafness , 2016, HCI.

[2]  A. Coenen,et al.  Mental effort affects vigilance enduringly: after-effects in EEG and behavior. , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  Lolita Rapolienė,et al.  The Reduction of Distress Using Therapeutic Geothermal Water Procedures in a Randomized Controlled Clinical Trial , 2015, Advances in preventive medicine.

[4]  J. Reason.  Human error: models and management , 2000, BMJ : British Medical Journal.

[5]  Kilseop Ryu,et al.  Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic , 2005 .

[6]  Raúl Rojas,et al.  Semi-autonomous Car Control Using Brain Computer Interfaces , 2012, IAS.

[7]  Scott A. Shappell,et al.  The Human Factors Analysis and Classification System : HFACS : final report. , 2000 .

[8]  Tzyy-Ping Jung,et al.  Biosensor Technologies for Augmented Brain–Computer Interfaces in the Next Decades , 2012, Proceedings of the IEEE.

[9]  R. Yerkes,et al.  The relation of strength of stimulus to rapidity of habit‐formation , 1908 .

[10]  Fabio Babiloni,et al.  Human Factors and Neurophysiological Metrics in Air Traffic Control: A Critical Review , 2017, IEEE Reviews in Biomedical Engineering.

[11]  Luca Longo,et al.  Inferential Models of Mental Workload with Defeasible Argumentation and Non-monotonic Fuzzy Reasoning: a Comparative Study , 2018, AI³@AI*IA.

[12]  Stefan Haufe,et al.  The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology , 2010, Front. Neurosci..

[13]  Fabio Babiloni,et al.  Monitoring Pilot's Cognitive Fatigue with Engagement Features in Simulated and Actual Flight Conditions Using an Hybrid fNIRS-EEG Passive BCI , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[14]  Laura Astolfi,et al.  Frontal EEG theta changes assess the training improvements of novices in flight simulation tasks , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[15]  Fabio Babiloni,et al.  EEG-based Approach-Withdrawal index for the pleasantness evaluation during taste experience in realistic settings , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[16]  Cyril R. Pernet,et al.  Misconceptions in the use of the General Linear Model applied to functional MRI: a tutorial for junior neuro-imagers , 2014, Front. Neurosci..

[17]  Febo Cincotti,et al.  Towards a multimodal bioelectrical framework for the online mental workload evaluation , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  M. Arango,et al.  Near-infrared spectroscopy as an index of brain and tissue oxygenation. , 2009, British journal of anaesthesia.

[19]  L. Giraudet,et al.  The neuroergonomic evaluation of human machine interface design in air traffic control using behavioral and EEG/ERP measures , 2015, Behavioural Brain Research.

[20]  Aidan Byrne,et al.  Mental Workload as an Outcome in Medical Education , 2017, H-WORKLOAD.

[21]  R. Gibberd,et al.  The Quality in Australian Health Care Study , 1995, The Medical journal of Australia.

[22]  D. Kirsh.  A Few Thoughts on Cognitive Overload , 2000 .

[23]  Gérard Dray,et al.  NIRS-measured prefrontal cortex activity in neuroergonomics: strengths and weaknesses , 2013, Front. Hum. Neurosci..

[24]  Paolo Fiorini,et al.  Neurophysiological measures for users' training objective assessment during simulated robot-assisted laparoscopic surgery , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[25]  S. Fairclough,et al.  The influence of performance feedback on goal-setting and mental effort regulation , 2009 .

[26]  Edward J Calabrese,et al.  Neuroscience and Hormesis: Overview and General Findings , 2008, Critical reviews in toxicology.

[27]  Fabio Babiloni,et al.  Passive BCI in Operational Environments: Insights, Recent Advances, and Future Trends , 2017, IEEE Transactions on Biomedical Engineering.

[28]  R. E. Richards,et al.  Structured methods for identifying and correcting potential human errors in space operations. , 1998, Acta astronautica.

[29]  Luca Longo,et al.  Analysing the Impact of Machine Learning to Model Subjective Mental Workload: A Case Study in Third-Level Education , 2018, H-WORKLOAD.

[30]  A. Wall,et al.  Book ReviewTo Err is Human: building a safer health system Kohn L T Corrigan J M Donaldson M S Washington DC USA: Institute of Medicine/National Academy Press ISBN 0 309 06837 1 $34.95 , 2000 .

[31]  Christophe Debruyne,et al.  On the Mental Workload Assessment of Uplift Mapping Representations in Linked Data , 2018, H-WORKLOAD.

[32]  Paul M. Salmon,et al.  Human Error and Road Transport: Phase One – Literature Review , 2005 .

[33]  Fabio Babiloni,et al.  A New Perspective for the Training Assessment: Machine Learning-Based Neurometric for Augmented User's Evaluation , 2017, Front. Neurosci..

[34]  Mark W. Scerbo,et al.  Theoretical Perspectives on Adaptive Automation , 2019, Human Performance in Automated and Autonomous Systems.

[35]  Nicholas J Ward,et al.  A systems analysis of the Ladbroke Grove rail crash. , 2005, Accident; analysis and prevention.

[36]  F. Thomas Eggemeier,et al.  Workload assessment methodology. , 1986 .

[37]  Luca Longo,et al.  Representing and Inferring Mental Workload via Defeasible Reasoning: A Comparison with the NASA Task Load Index and the Workload Profile , 2017, AI³@AI*IA.

[38]  Maria Chiara Leva,et al.  An Empirical Approach to Workload and Human Capability Assessment in a Manufacturing Plant , 2018, H-WORKLOAD.

[39]  Thorsten O. Zander,et al.  Utilizing Secondary Input from Passive Brain-Computer Interfaces for Enhancing Human-Machine Interaction , 2009, HCI.

[40]  Thibault Gateau,et al.  "Automation Surprise" in Aviation: Real-Time Solutions , 2015, CHI.

[41]  Fabio Babiloni,et al.  On the Use of Cognitive Neurometric Indexes in Aeronautic and Air Traffic Management Environments , 2015, Symbiotic.

[42]  R. Cabeza,et al.  Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies , 2000, Journal of Cognitive Neuroscience.

[43]  T. Brennan,et al.  INCIDENCE OF ADVERSE EVENTS AND NEGLIGENCE IN HOSPITALIZED PATIENTS , 2008 .

[44]  Fabio Babiloni,et al.  Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment , 2016, Front. Hum. Neurosci..

[45]  F. Freeman,et al.  A Closed-Loop System for Examining Psychophysiological Measures for Adaptive Task Allocation , 2000, The International journal of aviation psychology.

[46]  B Kirwan,et al.  Human error identification techniques for risk assessment of high risk systems--Part 1: Review and evaluation of techniques. , 1998, Applied ergonomics.

[47]  Fabio Babiloni,et al.  Correlation and Similarity between Cerebral and Non-Cerebral Electrical Activity for User’s States Assessment , 2019, Sensors.

[48]  R. Helmreich.  On error management: lessons from aviation , 2000, BMJ : British Medical Journal.

[49]  T. Jung,et al.  Dry and Noncontact EEG Sensors for Mobile Brain–Computer Interfaces , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[50]  Thibault Gateau,et al.  In silico vs. Over the Clouds: On-the-Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI , 2018, Front. Hum. Neurosci..

[51]  Robert Sargent,et al.  Development and evaluation of the Maintenance Error Decision Aid (MEDA) process , 2000 .

[52]  C. Vincent,et al.  Adverse events in British hospitals: preliminary retrospective record review , 2001, BMJ : British Medical Journal.

[53]  Daniel J. Barber,et al.  The Psychometrics of Mental Workload , 2015, Hum. Factors.

[54]  Fabio Babiloni,et al.  EEG-Based Cognitive Control Behaviour Assessment: an Ecological study with Professional Air Traffic Controllers , 2017, Scientific Reports.

[55]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[56]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[57]  Luca Longo,et al.  The Evolution of Cognitive Load Theory and the Measurement of Its Intrinsic, Extraneous and Germane Loads: A Review , 2018, H-WORKLOAD.

[58]  J. Grafman,et al.  Human prefrontal cortex: processing and representational perspectives , 2003, Nature Reviews Neuroscience.

[59]  A J Tattersall,et al.  An experimental evaluation of instantaneous self-assessment as a measure of workload. , 1996, Ergonomics.

[60]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[61]  Luca Longo,et al.  Experienced mental workload, perception of usability, their interaction and impact on task performance , 2018, PloS one.

[62]  Norbert Jaušovec,et al.  Working memory training: Improving intelligence – Changing brain activity , 2012, Brain and Cognition.

[63]  Barry H. Kantowitz,et al.  Human workload in aviation , 1988 .

[64]  Fabio Babiloni,et al.  Industrial Neuroscience in Aviation , 2017 .

[65]  Fabio Babiloni,et al.  EEG-Based Mental Workload Neurometric to Evaluate the Impact of Different Traffic and Road Conditions in Real Driving Settings , 2018, Front. Hum. Neurosci..

[66]  Luca Longo,et al.  Human Mental Workload: Models and Applications , 2017, Communications in Computer and Information Science.

[67]  Hasan Ayaz,et al.  Uav operators workload assessment by optical brain imaging technology (fnir) , 2015 .

[68]  Santosh Mathan,et al.  A joint human-automation cognitive system to support rapid decision-making in hostile environments , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[69]  Julien Penders,et al.  Wearable, Wireless EEG Solutions in Daily Life Applications: What are we Missing? , 2015, IEEE Journal of Biomedical and Health Informatics.

[70]  Fabio Babiloni,et al.  The Dry Revolution: Evaluation of Three Different EEG Dry Electrode Types in Terms of Signal Spectral Features, Mental States Classification and Usability , 2019, Sensors.

[71]  Fabio Babiloni,et al.  Gender and Age Related Effects While Watching TV Advertisements: An EEG Study , 2016, Comput. Intell. Neurosci..

[72]  B. Cain.  A Review of the Mental Workload Literature , 2007 .

[73]  K Rumar,et al.  The basic driver error: late detection. , 1990, Ergonomics.

[74]  Jonathan R. Wolpaw,et al.  Brain–Computer InterfacesPrinciples and Practice , 2012 .

[75]  Majid Fallahi,et al.  Assessment of operators’ mental workload using physiological and subjective measures in cement, city traffic and power plant control centers , 2016, Health promotion perspectives.

[76]  José del R. Millán,et al.  Improving Human Performance in a Real Operating Environment through Real-Time Mental Workload Detection , 2007 .

[77]  Chiara Leva,et al.  Human Performance Modelling for Adaptive Automation Journal of Physics: Conference Series , 2018, Journal of Physics: Conference Series.

[78]  Mica R. Endsley,et al.  Measurement of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[79]  Jens Rasmussen,et al.  Human errors. a taxonomy for describing human malfunction in industrial installations , 1982 .

[80]  Chris Berka,et al.  Real-Time Analysis of EEG Indexes of Alertness, Cognition, and Memory Acquired With a Wireless EEG Headset , 2004, Int. J. Hum. Comput. Interact..

[81]  F. T. Eggemeier,et al.  Recommendations for Mental Workload Measurement in a Test and Evaluation Environment , 1993 .

[82]  Jialin Fan,et al.  The Impact of Workload and Fatigue on Performance , 2017, H-WORKLOAD.

[83]  Tamsyn Edwards,et al.  The Relationship between Workload and Performance in Air Traffic Control , 2017, H-WORKLOAD.

[84]  Dov Eden,et al.  The inverted-U relationship between stress and performance: A field study , 1996 .

[85]  Glenn F. Wilson,et al.  Psychophysiological responses to changes in workload during simulated air traffic control , 1996, Biological Psychology.

[86]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[87]  Joel S. Warm,et al.  Vigilance Requires Hard Mental Work and Is Stressful , 2008, Hum. Factors.

[88]  Mark S. Young,et al.  Predicting design induced pilot error using HET (human error template) – A new formal human error identification method for flight decks , 2006, The Aeronautical Journal (1968).

[89]  S Pozzi,et al.  A passive brain-computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks. , 2016, Progress in brain research.

[90]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[91]  Febo Cincotti,et al.  Hybrid P300-based brain-computer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task. , 2015, Archives of physical medicine and rehabilitation.

[92]  J. VanMeter,et al.  Event-related fast optical signal in a rapid object recognition task: Improving detection by the independent component analysis , 2008, Brain Research.

[93]  Majid Fallahi,et al.  Effects of mental workload on physiological and subjective responses during traffic density monitoring: A field study. , 2016, Applied ergonomics.

[94]  Daniel Sánchez Morillo,et al.  Dry EEG Electrodes , 2014, Sensors.

[95]  Aidan Byrne,et al.  Measurement of Mental Workload in Clinical Medicine: A Review Study , 2011, Anesthesiology and pain medicine.

[96]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[97]  Angel Jiménez Molina,et al.  Using Psychophysiological Sensors to Assess Mental Workload During Web Browsing , 2018, Sensors.

[98]  Christopher D. Wickens,et al.  Mental Workload: Assessment, Prediction and Consequences , 2017, H-WORKLOAD.

[99]  Fabio Babiloni,et al.  Quantitative Assessment of the Training Improvement in a Motor-Cognitive Task by Using EEG, ECG and EOG Signals , 2015, Brain Topography.

[100]  M Doppelmayr,et al.  Brain oscillations and human memory: EEG correlates in the upper alpha and theta band , 1997, Neuroscience Letters.

[101]  Luca Longo.  Designing Medical Interactive Systems Via Assessment of Human Mental Workload , 2015, 2015 IEEE 28th International Symposium on Computer-Based Medical Systems.

[102]  Aaron M. Novstrup,et al.  Assessing Workload in Human-Machine Teams from Psychophysiological Data with Sparse Ground Truth , 2018, H-WORKLOAD.

[103]  J. Gorman,et al.  Effect of mental stress throughout the day on cardiac autonomic control , 1994, Biological Psychology.

[104]  Ulf Ahlstrom,et al.  Portable Weather Applications for General Aviation Pilots , 2016, Hum. Factors.

[105]  Fabio Babiloni,et al.  Alpha and Theta EEG Variations as Indices of Listening Effort to Be Implemented in Neurofeedback Among Cochlear Implant Users , 2017, Symbiotic.

[106]  Guofa Shou,et al.  Probing neural activations from continuous EEG in a real-world task: Time-frequency independent component analysis , 2012, Journal of Neuroscience Methods.

[107]  Raja Parasuraman,et al.  Neuroergonomics: Research and practice , 2003 .

[108]  A. Owen,et al.  Anterior prefrontal cortex: insights into function from anatomy and neuroimaging , 2004, Nature Reviews Neuroscience.

[109]  Christian Mühl,et al.  EEG-based workload estimation across affective contexts , 2014, Front. Neurosci..

[110]  Fabio Babiloni,et al.  Investigating Driver Fatigue versus Alertness Using the Granger Causality Network , 2015, Sensors.

[111]  Daniel Gopher,et al.  Workload: An examination of the concept. , 1986 .

[112]  Barry Kirwan,et al.  Development and application of a human error identification tool for air traffic control. , 2002, Applied ergonomics.

[113]  Robert J Barry,et al.  Arousal and activation effects on physiological and behavioral responding during a continuous performance task. , 2007, Acta neurobiologiae experimentalis.

[114]  Daphne N. Yu,et al.  High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice. , 1997, Cerebral cortex.

[115]  T. Brennan,et al.  The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. , 1991, The New England journal of medicine.

[116]  A. Colosimo,et al.  EEG Frontal Asymmetry Related to Pleasantness of Olfactory Stimuli in Young Subjects , 2016 .

[117]  Klaus-Robert Müller,et al.  Machine learning for real-time single-trial EEG-analysis: From brain–computer interfacing to mental state monitoring , 2008, Journal of Neuroscience Methods.

[118]  J. Sexton,et al.  Error, stress, and teamwork in medicine and aviation: cross sectional surveys , 2000, BMJ : British Medical Journal.

[119]  F. Babiloni,et al.  A covert attention P300-based brain–computer interface: Geospell , 2012, Ergonomics.

[120]  L. Åberg,et al.  Dimensions of aberrant driver behaviour. , 1998, Ergonomics.

[121]  Dylan D. Schmorrow,et al.  Enhancing Mitigation in Augmented Cognition , 2007 .

[122]  Peter A. Hancock,et al.  ACTIVE AND PASSIVE FATIGUE STATES , 2001 .

[123]  F. Babiloni,et al.  Neuroelectrical Indexes for the Study of the Efficacy of TV Advertising Stimuli , 2016 .

[124]  Jens Rasmussen,et al.  The Definition of Human Error and a Taxonomy for Technical System Design , 1987 .

[125]  I. Parberry,et al.  Evaluating player task engagement and arousal using electroencephalography , 2015 .

[126]  Wolfgang Rosenstiel,et al.  Cognitive state monitoring and the design of adaptive instruction in digital environments: lessons learned from cognitive workload assessment using a passive brain-computer interface approach , 2014, Front. Neurosci..

[127]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.