Human Mental Workload: Models and Applications

Mental workload (MWL) measurement is a complex multidisciplinary research field. In the last 50 years of research endeavour, MWL measurement has mainly produced theory-driven models. Some of the reasons for justifying this trend includes the omnipresent uncertainty about how to define the construct of MWL and the limited use of data-driven research methodologies. This work presents novel research focused on the investigation of the capability of a selection of supervised Machine Learning (ML) classification techniques to produce data-driven computational models of MWL for the prediction of objective performance. These are then compared to two state-of-the-art subjective techniques for the assessment of MWL, namely the NASA Task Load Index and the Workload Profile, through an analysis of their concurrent and convergent validity. Findings show that the data-driven models generally tend to outperform the two baseline selected techniques.

[1]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[2]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[3]  Amandalynne Paullada,et al.  ATHENA - A Zero-Intrusion No Contact Method for Workload Detection Using Linguistics, Keyboard Dynamics, and Computer Vision , 2016, HCI.

[4]  Luca Longo,et al.  Cognitive Effort for Multi-agent Systems , 2010, Brain Informatics.

[5]  Sam Cromie,et al.  A Dynamic Task Representation Method for a Virtual Reality Application , 2009, HCI.

[6]  Fabio Rusconi,et al.  The Importance of Human Mental Workload in Web Design , 2012, WEBIST.

[7]  Daniel Afergan,et al.  Using fNIRS for Real-Time Cognitive Workload Assessment , 2016, HCI.

[8]  Gerhard Tröster,et al.  Monitoring of mental workload levels during an everyday life office-work scenario , 2013, Personal and Ubiquitous Computing.

[9]  Gary E. Birch,et al.  Comparison of Evaluation Metrics in Classification Applications with Imbalanced Datasets , 2008, 2008 Seventh International Conference on Machine Learning and Applications.

[10]  Luca Longo,et al.  Modeling Mental Workload Via Rule-Based Expert System: A Comparison with NASA-TLX and Workload Profile , 2016, AIAI.

[11]  Susana Rubio,et al.  Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA‐TLX, and Workload Profile Methods , 2004 .

[12]  Eric N. Wiebe,et al.  An examination of two mental workload measurement approaches to understanding multimedia learning , 2010, Comput. Hum. Behav..

[13]  P. Tsang,et al.  Diagnosticity and multidimensional subjective workload ratings. , 1996, Ergonomics.

[14]  Glenn Fung,et al.  On the Dangers of Cross-Validation. An Experimental Evaluation , 2008, SDM.

[15]  F. Thomas Eggemeier,et al.  Workload assessment methodology. , 1986 .

[16]  Michael P. Jenkins,et al.  Are Behavioral Measures Useful for Detecting Cognitive Workload During Human-Computer Interaction? , 2017 .

[17]  Joseph T. Coyne,et al.  Cognitive context detection in UAS operators using eye-gaze patterns on computer screens , 2016, SPIE Defense + Security.

[18]  Chumphol Bunkhumpornpat,et al.  DBSMOTE: Density-Based Synthetic Minority Over-sampling TEchnique , 2011, Applied Intelligence.

[19]  Michael P. Cohen,et al.  Stratified Sampling , 2022, The SAGE Encyclopedia of Research Design.

[20]  Luca Longo,et al.  A Computational Analysis of Cognitive Effort , 2010, ACIIDS.

[21]  Klaus-Robert Müller,et al.  Classifying Single Trial EEG: Towards Brain Computer Interfacing , 2001, NIPS.

[22]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[23]  Klaus-Robert Müller,et al.  Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and multiclass paradigms , 2004, IEEE Transactions on Biomedical Engineering.

[24]  Gavin C. Cawley,et al.  On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation , 2010, J. Mach. Learn. Res..

[25]  Luca Longo,et al.  On the Relationship between Perception of Usability and Subjective Mental Workload of Web Interfaces , 2015, 2015 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT).

[26]  Fumio Mizoguchi,et al.  Classifying Cognitive Load and Driving Situation with Machine Learning , 2014 .

[27]  D. Tax,et al.  Feature scaling in support vector data description , 2002 .

[28]  Ronald H. Stevens,et al.  Integrating EEG Models of Cognitive Load with Machine Learning Models of Scientific Problem Solving , 2013 .

[29]  Robert P. W. Duin,et al.  Feature Scaling in Support Vector Data Descriptions , 2000 .

[30]  Norbert Stoll,et al.  Fuzzy Techniques for Subjective Workload-Score Modeling Under Uncertainties , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Luca Longo,et al.  Formalising human mental workload as non-monotonic concept for adaptive and personalised web-design , 2012, UMAP.

[32]  Anirban Bhowal,et al.  Comparative analysis of machine learning algorithms along with classifiers for network intrusion detection , 2015, 2015 International Conference on Smart Technologies and Management for Computing, Communication, Controls, Energy and Materials (ICSTM).

[33]  Luca Longo,et al.  A defeasible reasoning framework for human mental workload representation and assessment , 2015, Behav. Inf. Technol..

[34]  Oliver Sträter,et al.  Human Error Prediction in ATM via Cognitive Simulation: Preliminary Study (PSAM-0268) , 2006 .

[35]  Luca Longo,et al.  Human-Computer Interaction and Human Mental Workload: Assessing Cognitive Engagement in the World Wide Web , 2011, INTERACT.

[36]  Thierry Baccino,et al.  Approximation of on-line mental workload index in ATC simulated multitasks , 2010 .

[37]  Thao Thanh Pham,et al.  Sparse fNIRS Feature Estimation via Unsupervised Learning for Mental Workload Classification , 2015, Advances in Neural Networks.

[38]  Luca Longo.  Designing Medical Interactive Systems Via Assessment of Human Mental Workload , 2015, 2015 IEEE 28th International Symposium on Computer-Based Medical Systems.

[39]  Jing Zhang,et al.  Driver cognitive workload estimation: a data-driven perspective , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[40]  B. Cain.  A Review of the Mental Workload Literature , 2007 .

[41]  Desney S. Tan,et al.  Using a low-cost electroencephalograph for task classification in HCI research , 2006, UIST.

[42]  Luca Longo,et al.  Mental Workload in Medicine: Foundations, Applications, Open Problems, Challenges and Future Perspectives , 2016, 2016 IEEE 29th International Symposium on Computer-Based Medical Systems (CBMS).

[43]  Arie Ben-David,et al.  About the relationship between ROC curves and Cohen's kappa , 2008, Eng. Appl. Artif. Intell..

[44]  Matthias Scheutz,et al.  Brainput: enhancing interactive systems with streaming fnirs brain input , 2012, CHI.

[45]  Henk G. Stassen,et al.  Internal representation, internal model, human performance model and mental workload , 1988, Autom..

[46]  Saturnino Luz,et al.  Predicting Cognitive Load Levels from Speech Data , 2016, Recent Advances in Nonlinear Speech Processing.

[47]  Thomas E. Nygren,et al.  The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload , 1988 .

[48]  Luca Longo,et al.  Formalising Human Mental Workload as a Defeasible Computational Concept , 2014 .

[49]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[50]  Ramesh Raskar,et al.  Workload Assessment with eye Movement Monitoring Aided by Non-invasive and Unobtrusive Micro-fabricated Optical Sensors , 2015, UIST.