Assessing cognitive mental workload via EEG signals and an ensemble deep learning classifier based on denoising autoencoders

To estimate the reliability and cognitive states of operator performance in a human-machine collaborative environment, we propose a novel human mental workload (MW) recognizer based on deep learning principles and utilizing the features of the electroencephalogram (EEG). To determine personalized properties in high dimensional EEG indicators, we introduce a feature mapping layer in stacked denoising autoencoder (SDAE) that is capable of preserving the local information in EEG dynamics. The ensemble classifier is then built via the subject-specific integrated deep learning committee, and adapts to the cognitive properties of a specific human operator and alleviates inter-subject feature variations. We validate our algorithms and the ensemble SDAE classifier with local information preservation (denoted by EL-SDAE) on an EEG database collected during the execution of complex human-machine tasks. The classification performance indicates that the EL-SDAE outperforms several classical MW estimators when its optimal network architecture has been identified.

[1]  Nilanjan Sarkar,et al.  EEG-Based Affect and Workload Recognition in a Virtual Driving Environment for ASD Intervention , 2018, IEEE Transactions on Biomedical Engineering.

[2]  Xiqing Cui,et al.  Identification of Mental Workload Using Imbalanced EEG Data and DySMOTE-based Neural Network Approach , 2016 .

[3]  Liqing Zhang,et al.  Feature learning from incomplete EEG with denoising autoencoder , 2014, Neurocomputing.

[4]  Carryl L. Baldwin,et al.  Adaptive training using an artificial neural network and EEG metrics for within- and cross-task workload classification , 2012, NeuroImage.

[5]  Jianping Li,et al.  A deep learning ensemble approach for crude oil price forecasting , 2017 .

[6]  Jonathan M. Garibaldi,et al.  Modeling and control of operator functional state in a unified framework of fuzzy inference petri nets , 2017, Comput. Methods Programs Biomed..

[7]  Ya Ju Fan,et al.  Autoencoder Node Saliency: Selecting Relevant Latent Representations , 2017, Pattern Recognit..

[8]  Wei Liu,et al.  Detecting driving fatigue with multimodal deep learning , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[9]  Benjamin Brooks,et al.  Measuring mental workload and physiological reactions in marine pilots: Building bridges towards redlines of performance. , 2018, Applied ergonomics.

[10]  Yong Wang,et al.  Matrix exponential based discriminant locality preserving projections for feature extraction , 2018, Neural Networks.

[11]  Shiqi Huang,et al.  Locality Preserving Projection Based on Endmember Extraction for Hyperspectral Image Dimensionality Reduction and Target Detection , 2016, Applied spectroscopy.

[12]  Chris H. Q. Ding,et al.  A Nonnegative Locally Linear KNN model for image recognition , 2018, Pattern Recognit..

[13]  Anirban Mukherjee,et al.  A deep learning approach to fetal-ECG signal reconstruction , 2016, 2016 Twenty Second National Conference on Communication (NCC).

[14]  Luay Fraiwan,et al.  Neonatal sleep state identification using deep learning autoencoders , 2017, 2017 IEEE 13th International Colloquium on Signal Processing & its Applications (CSPA).

[15]  Zhong Yin,et al.  Cross-session classification of mental workload levels using EEG and an adaptive deep learning model , 2017, Biomed. Signal Process. Control..

[16]  Shunping Xie,et al.  Representative splitting cross validation , 2018, Chemometrics and Intelligent Laboratory Systems.

[17]  Dick de Waard,et al.  Classifying visuomotor workload in a driving simulator using subject specific spatial brain patterns , 2013, Front. Neurosci..

[18]  Yufeng Ke,et al.  Towards an effective cross-task mental workload recognition model using electroencephalography based on feature selection and support vector machine regression. , 2015, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[19]  Li Wen,et al.  Dimensionality reduction on Anchorgraph with an efficient Locality Preserving Projection , 2016, Neurocomputing.

[20]  Olga Sourina,et al.  EEG-based mental workload recognition related to multitasking , 2015, 2015 10th International Conference on Information, Communications and Signal Processing (ICICS).

[21]  Hasan Ayaz,et al.  Optical brain monitoring for operator training and mental workload assessment , 2012, NeuroImage.

[22]  Kaspar Althoefer,et al.  Knock-Knock: Acoustic object recognition by using stacked denoising autoencoders , 2017, Neurocomputing.

[23]  Miyoung Kim,et al.  Comparative analysis of cognitive tasks for modeling mental workload with electroencephalogram , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[24]  Xiaoqing Feng,et al.  Multimodal video classification with stacked contractive autoencoders , 2016, Signal Process..

[25]  Yang Yang,et al.  Supervised feature learning via l2-norm regularized logistic regression for 3D object recognition , 2015, Neurocomputing.

[26]  U. Rajendra Acharya,et al.  Deep learning for healthcare applications based on physiological signals: A review , 2018, Comput. Methods Programs Biomed..

[27]  Nigel H. Lovell,et al.  Beyond Subjective Self-Rating: EEG Signal Classification of Cognitive Workload , 2015, IEEE Transactions on Autonomous Mental Development.

[28]  Olga Sourina,et al.  EEG-based Mental Workload and Stress Recognition of Crew Members in Maritime Virtual Simulator: A Case Study , 2017, 2017 International Conference on Cyberworlds (CW).

[29]  Raja Parasuraman,et al.  Display Integration Enhances Information Sampling and Decision Making in Automated Fault Management in a Simulated Spaceflight Micro-World , 2002, Aviation, space, and environmental medicine.

[30]  Nadine Matton,et al.  Using theta and alpha band power to assess cognitive workload in multitasking environments. , 2018, International Journal of Psychophysiology.

[31]  Wei Zhang,et al.  Two-stage ELM for phishing Web pages detection using hybrid features , 2017, World Wide Web.

[32]  Ahmet Omurtag,et al.  Assessment of mental workload by EEG+FNIRS , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[33]  Jakub M. Tomczak,et al.  Interaction prediction in structure-based virtual screening using deep learning , 2018, Comput. Biol. Medicine.

[34]  J Sauer,et al.  A conceptual framework for designing micro-worlds for complex work domains: a case study of the Cabin Air Management System. , 2000, Computers in human behavior.

[35]  Suman K. Mitra,et al.  On some variants of locality preserving projection , 2016, Neurocomputing.

[36]  Rubin Wang,et al.  Recognition of Mental Workload Levels Under Complex Human–Machine Collaboration by Using Physiological Features and Adaptive Support Vector Machines , 2015, IEEE Transactions on Human-Machine Systems.

[37]  Jun Wang,et al.  An intelligent diagnosis scheme based on generative adversarial learning deep neural networks and its application to planetary gearbox fault pattern recognition , 2018, Neurocomputing.

[38]  Majid Fallahi,et al.  Analysis of the mental workload of city traffic control operators while monitoring traffic density: A field study , 2016 .

[39]  Tiberius Dumitriu,et al.  A Comparative Study on Classification of Working Memory Tasks Using EEG Signals , 2017, 2017 21st International Conference on Control Systems and Computer Science (CSCS).

[40]  Xiwang Li,et al.  Using an ensemble machine learning methodology-Bagging to predict occupants’ thermal comfort in buildings , 2018, Energy and Buildings.

[41]  Peter Chapman,et al.  Mental workload is reflected in driver behaviour, physiology, eye movements and prefrontal cortex activation. , 2018, Applied ergonomics.

[42]  Xinyi Song,et al.  Revealing the "invisible Gorilla" in construction: Estimating construction safety through mental workload assessment , 2016 .

[43]  Akin Ozçift,et al.  Random forests ensemble classifier trained with data resampling strategy to improve cardiac arrhythmia diagnosis. , 2011, Computers in biology and medicine.

[44]  Febo Cincotti,et al.  Towards a multimodal bioelectrical framework for the online mental workload evaluation , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[45]  Barbara Deml,et al.  Electrocardiographic features for the measurement of drivers' mental workload. , 2017, Applied ergonomics.

[46]  Yanchun Zhang,et al.  Epileptic seizure detection in EEG signals using tunable-Q factor wavelet transform and bootstrap aggregating , 2016, Comput. Methods Programs Biomed..

[47]  Jun Zhang,et al.  Dynamic frequency feature selection based approach for classification of motor imageries , 2016, Comput. Biol. Medicine.

[48]  Olga Sourina,et al.  CogniMeter: EEG-based Emotion, Mental Workload and Stress Visual Monitoring , 2015, 2015 International Conference on Cyberworlds (CW).

[49]  Moon Kyoung Choi,et al.  Development of an EEG-based workload measurement method in nuclear power plants , 2018 .

[50]  Jacek Gwizdka,et al.  Using Wireless EEG Signals to Assess Memory Workload in the $n$-Back Task , 2016, IEEE Transactions on Human-Machine Systems.

[51]  Chen Lu,et al.  Fault diagnosis of rotary machinery components using a stacked denoising autoencoder-based health state identification , 2017, Signal Process..