A Novel Mutual Information Based Feature Set for Drivers’ Mental Workload Evaluation Using Machine Learning

Analysis of physiological signals, electroencephalography more specifically, is considered a very promising technique to obtain objective measures for mental workload evaluation, however, it requires a complex apparatus to record, and thus, with poor usability in monitoring in-vehicle drivers’ mental workload. This study proposes a methodology of constructing a novel mutual information-based feature set from the fusion of electroencephalography and vehicular signals acquired through a real driving experiment and deployed in evaluating drivers’ mental workload. Mutual information of electroencephalography and vehicular signals were used as the prime factor for the fusion of features. In order to assess the reliability of the developed feature set mental workload score prediction, classification and event classification tasks were performed using different machine learning models. Moreover, features extracted from electroencephalography were used to compare the performance. In the prediction of mental workload score, expert-defined scores were used as the target values. For classification tasks, true labels were set from contextual information of the experiment. An extensive evaluation of every prediction tasks was carried out using different validation methods. In predicting the mental workload score from the proposed feature set lowest mean absolute error was 0.09 and for classifying mental workload highest accuracy was 94%. According to the outcome of the study, it can be stated that the novel mutual information based features developed through the proposed approach can be employed to classify and monitor in-vehicle drivers’ mental workload.

[1]  R. Elul Gaussian Behavior of the Electroencephalogram: Changes during Performance of Mental Task , 1969, Science.

[2]  Fabio Babiloni,et al.  EEG-Based Mental Workload Neurometric to Evaluate the Impact of Different Traffic and Road Conditions in Real Driving Settings , 2018, Front. Hum. Neurosci..

[3]  Fabio Babiloni,et al.  A New Perspective for the Training Assessment: Machine Learning-Based Neurometric for Augmented User's Evaluation , 2017, Front. Neurosci..

[4]  Pavlo D. Antonenko The effect of leads on cognitive load and learning in a conceptually rich hypertext environment , 2007 .

[5]  Pan Wang,et al.  Using Support Vector Machine on EEG for Advertisement Impact Assessment , 2018, Front. Neurosci..

[6]  Fabio Babiloni,et al.  Brain–Computer Interface-Based Adaptive Automation to Prevent Out-Of-The-Loop Phenomenon in Air Traffic Controllers Dealing With Highly Automated Systems , 2019, Front. Hum. Neurosci..

[7]  Velanganny Cyrilraj,et al.  A vehicle control system using a time synchronized Hybrid VANET to reduce road accidents caused by human error , 2016, Veh. Commun..

[8]  Xinwei Deng,et al.  Experimental design , 2012, WIREs Data Mining Knowl. Discov..

[9]  Christopher D. Wickens,et al.  Attention-Situation Awareness (A-SA) Model of Pilot Error , 2007 .

[10]  David A. Freedman,et al.  Statistical Models: Theory and Practice: References , 2005 .

[11]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[12]  Pete Thomas,et al.  Identifying the causes of road crashes in Europe. , 2013, Annals of advances in automotive medicine. Association for the Advancement of Automotive Medicine. Annual Scientific Conference.

[13]  Shaibal Barua,et al.  Multivariate Data Analytics to Identify Driver's Sleepiness, Cognitive load, and Stress , 2019 .

[14]  Mobyen Uddin Ahmed,et al.  In-Vehicle Stress Monitoring Based on EEG Signal , 2017 .

[15]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[16]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[17]  W B Verwey,et al.  On-line driver workload estimation. Effects of road situation and age on secondary task measures , 2000, Ergonomics.

[18]  Bryan Reimer,et al.  Classifying driver workload using physiological and driving performance data: two field studies , 2014, CHI.

[19]  Fabio Babiloni,et al.  Passive BCI in Operational Environments: Insights, Recent Advances, and Future Trends , 2017, IEEE Transactions on Biomedical Engineering.

[20]  Fabio Babiloni,et al.  EEG-Based Mental Workload Assessment During Real Driving , 2019, Neuroergonomics.

[21]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[22]  Motoki Sakai Kernel Nonnegative Matrix Factorization with Constraint Increasing the Discriminability of Two Classes for the EEG Feature Extraction , 2013, 2013 International Conference on Signal-Image Technology & Internet-Based Systems.

[23]  Gennaro Nicola Bifulco,et al.  Validity of Mental Workload Measures in a Driving Simulation Environment , 2018, Journal of Advanced Transportation.

[24]  S. Eckstein Ethical principles for medical research involving human subjects. , 2001, European journal of emergency medicine : official journal of the European Society for Emergency Medicine.

[25]  Francesco Carlo Morabito,et al.  On the Classification of EEG Signal by Using an SVM Based Algorithm , 2018, Multidisciplinary Approaches to Neural Computing.

[26]  Shaibal Barua,et al.  EEG sensor based classification for assessing psychological stress. , 2013, Studies in health technology and informatics.

[27]  Saturnino Luz,et al.  Human Mental Workload: Models and Applications , 2017, Communications in Computer and Information Science.

[28]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[29]  Fabio Babiloni,et al.  A new regression-based method for the eye blinks artifacts correction in the EEG signal, without using any EOG channel , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[30]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[31]  Lisbeth Harms,et al.  Variation in drivers' cognitive load. Effects of driving through village areas and rural junctions , 1991 .

[32]  Matthias Rötting,et al.  Influence of Task Combination on EEG Spectrum Modulation for Driver Workload Estimation , 2011, Hum. Factors.

[33]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[34]  S Pozzi,et al.  A passive brain-computer interface application for the mental workload assessment on professional air traffic controllers during realistic air traffic control tasks. , 2016, Progress in brain research.

[35]  Michael E. Smith,et al.  Monitoring Working Memory Load during Computer-Based Tasks with EEG Pattern Recognition Methods , 1998, Hum. Factors.

[36]  Travis E. Oliphant,et al.  Guide to NumPy , 2015 .

[37]  Fabio Babiloni,et al.  Adaptive Automation Triggered by EEG-Based Mental Workload Index: A Passive Brain-Computer Interface Application in Realistic Air Traffic Control Environment , 2016, Front. Hum. Neurosci..

[38]  F. Sherwani,et al.  Wavelet based feature extraction for classification of motor imagery signals , 2016, 2016 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES).

[39]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[40]  Catherine Berthelon,et al.  Mental workload and driving , 2014, Front. Psychol..

[41]  Wolfgang Fastenmeier,et al.  Driving Task Analysis as a Tool in Traffic Safety Research and Practice , 2007 .

[42]  Dick de Waard,et al.  Monitoring drivers' mental workload in driving simulators using physiological measures. , 2010, Accident; analysis and prevention.

[43]  Mobyen Uddin Ahmed,et al.  Automated EEG Artifact Handling With Application in Driver Monitoring , 2018, IEEE Journal of Biomedical and Health Informatics.

[44]  Fabio Babiloni,et al.  EEG-Based Cognitive Control Behaviour Assessment: an Ecological study with Professional Air Traffic Controllers , 2017, Scientific Reports.

[45]  Phillip M. Alday,et al.  Towards a reliable, automated method of individual alpha frequency (IAF) quantification , 2017, bioRxiv.

[46]  Zhang Jianhua,et al.  Recognition of Cognitive Task Load levels using single channel EEG and Stacked Denoising Autoencoder , 2016, 2016 35th Chinese Control Conference (CCC).

[47]  Aamir Saeed Malik,et al.  EEG-based brain functional connectivity during distracted driving , 2015, 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[48]  Tingxi Wen,et al.  Deep Convolution Neural Network and Autoencoders-Based Unsupervised Feature Learning of EEG Signals , 2018, IEEE Access.

[49]  Michael E. Smith,et al.  Neurophysiological measures of cognitive workload during human-computer interaction , 2003 .

[50]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[51]  Aamir Saeed Malik,et al.  Discriminating the different human brain states with EEG signals using Fractal dimension: A nonlinear approach , 2014, 2014 IEEE International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA).

[52]  S. Kar,et al.  EEG signal analysis for the assessment and quantification of driver’s fatigue , 2010 .

[53]  Przemyslaw Guzik,et al.  ECG by mobile technologies. , 2016, Journal of electrocardiology.

[54]  Mobyen Uddin Ahmed,et al.  Classifying Drivers' Cognitive Load Using EEG Signals , 2017, pHealth.

[55]  Debasis Samanta,et al.  Classification of EEG Signals for Cognitive Load Estimation Using Deep Learning Architectures , 2018, IHCI.

[56]  Fátima Pereira da Silva Mental Workload, Task Demand and Driving Performance: What Relation?☆ , 2014 .

[57]  Hamidur Rahman,et al.  Non-contact-based driver's cognitive load classification using physiological and vehicular parameters , 2020, Biomed. Signal Process. Control..

[58]  Christiane,et al.  World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. , 2004, Journal international de bioethique = International journal of bioethics.

[59]  Xavier Robin,et al.  pROC: an open-source package for R and S+ to analyze and compare ROC curves , 2011, BMC Bioinformatics.

[60]  Wes McKinney,et al.  Data Structures for Statistical Computing in Python , 2010, SciPy.

[61]  Fabio Babiloni,et al.  On the Use of Cognitive Neurometric Indexes in Aeronautic and Air Traffic Management Environments , 2015, Symbiotic.

[62]  L. McEvoy,et al.  Neurophysiological indices of strategy development and skill acquisition. , 1999, Brain research. Cognitive brain research.

[63]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[64]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[65]  P. Geethanjali,et al.  Time domain Feature extraction and classification of EEG data for Brain Computer Interface , 2012, 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery.

[66]  D. de Waard,et al.  Handbook of driving simulation for engineering, medicine and psychology , 2011 .

[67]  Gianluca Di Flumeri,et al.  Deep Learning for Automatic EEG Feature Extraction: An Application in Drivers' Mental Workload Classification , 2019, H-WORKLOAD.

[68]  Shigeki Sugano,et al.  Multiclass Classification of Driver Perceived Workload Using Long Short-Term Memory based Recurrent Neural Network , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[69]  Woojin Kim,et al.  A study on the cognitive workload characteristics according to the driving behavior in the urban road , 2018, 2018 International Conference on Electronics, Information, and Communication (ICEIC).

[70]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[71]  D. Song,et al.  EEG Based Emotion Identification Using Unsupervised Deep Feature Learning , 2015 .

[72]  Jim Nixon,et al.  Measuring mental workload using physiological measures: A systematic review. , 2019, Applied ergonomics.

[73]  Paolo Fiorini,et al.  EEG-Based Workload Index as a Taxonomic Tool to Evaluate the Similarity of Different Robot-Assisted Surgery Systems , 2019, H-WORKLOAD.