Quantifying Cognitive Workload in Simulated Flight Using Passive, Dry EEG Measurements

A reliable method for quantifying cognitive workload in pilots could find uses in flight training and scheduling, cockpit design, and improving flight safety. Many proposed methods for monitoring cognitive workload in this population rely on measuring physiological responses to externally delivered probe stimuli and/or use traditional gel-based electroencephalography (EEG) sensors. Here we develop passive, probe-independent algorithms for classifying three levels of flight task complexity based on 4-channel, gel-free EEG during simulated flight. Using a library of 168 input features drawn from different data science application domains, we evaluated 13 different classifiers, using a nested tenfold cross-validation procedure to estimate generalization performance. The best subsets of features yielded a median classification accuracy of 90.17% across subjects, with perfect accuracy in one subject and greater than 75% in 16 of 21 subjects. Though EEG line length and linear discriminant analysis were generally among the most effective features and classifiers, respectively, we find that to maximize prediction accuracy, feature set-classifier combinations should be individualized. No single channel proved more valuable than another in predicting flight task complexity, but combining EEG features across channels maintained or improved performance in 81% of subjects.

[1]  A. Murata,et al.  Assessment of mental fatigue during VDT task using event-related potential (P300) , 2000, Proceedings 9th IEEE International Workshop on Robot and Human Interactive Communication. IEEE RO-MAN 2000 (Cat. No.00TH8499).

[2]  Caroline Dussault,et al.  EEG and ECG changes during selected flight sequences. , 2004, Aviation, space, and environmental medicine.

[3]  Louise Venables,et al.  The influence of task demand and learning on the psychophysiological response. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  Hankins Tc,et al.  A comparison of heart rate, eye activity, EEG and subjective measures of pilot mental workload during flight. , 1998, Aviation, space, and environmental medicine.

[5]  F. Fleuret Fast Binary Feature Selection with Conditional Mutual Information , 2004, J. Mach. Learn. Res..

[6]  Matthew W. Miller,et al.  A novel approach to the physiological measurement of mental workload. , 2011, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[7]  Zoly J. Koles,et al.  Mental activity and the e.e.g.: Task and workload related effects , 2006, Medical and Biological Engineering and Computing.

[8]  Rodolphe J. Gentili,et al.  Cerebral-cortical networking and activation increase as a function of cognitive-motor task difficulty , 2012, Biological Psychology.

[9]  James Bailey,et al.  Reconsidering Mutual Information Based Feature Selection: A Statistical Significance View , 2014, AAAI.

[10]  John H. L. Hansen,et al.  Nonlinear feature based classification of speech under stress , 2001, IEEE Trans. Speech Audio Process..

[11]  Richard Simon,et al.  Bias in error estimation when using cross-validation for model selection , 2006, BMC Bioinformatics.

[12]  J. F. Kaiser,et al.  On a simple algorithm to calculate the 'energy' of a signal , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[13]  Begoña Garcia-Zapirain,et al.  EEG artifact removal—state-of-the-art and guidelines , 2015, Journal of neural engineering.

[14]  Dick de Waard,et al.  Monitoring drivers' mental workload in driving simulators using physiological measures. , 2010, Accident; analysis and prevention.

[15]  Michael K. Johnson,et al.  Probe-independent EEG assessment of mental workload in pilots , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[16]  W. Pritchard Psychophysiology of P300. , 1981, Psychological bulletin.

[17]  C. Guézennec,et al.  EEG and ECG changes during simulator operation reflect mental workload and vigilance. , 2005, Aviation, space, and environmental medicine.

[18]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.

[19]  Nadine Eberhardt,et al.  Bioelectrical Signal Processing In Cardiac And Neurological Applications , 2016 .

[20]  Maan M. Shaker,et al.  EEG Waves Classifier using Wavelet Transform and Fourier Transform , 2007 .

[21]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[22]  Thanh An Nguyen,et al.  A PILOT STUDY TO ASSESS DESIGNER'S MENTAL STRESS USING EYE GAZE SYSTEM AND ELECTROENCEPHALOGRAM , 2009 .

[23]  Robert J. K. Jacob,et al.  DISCRIMINATION OF MENTAL WORKLOAD LEVELS IN HUMAN SUBJECTS WITH FUNCTIONAL NEAR-INFRARED SPECTROSCOPY , 2008 .

[24]  Tzyy-Ping Jung,et al.  A brain-machine interface using dry-contact, low-noise EEG sensors , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[25]  Guofa Shou,et al.  Investigation of independent components based EEG metrics for mental fatigue in simulated ATC task , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[26]  Rodolphe J. Gentili,et al.  Brain biomarkers based assessment of cognitive workload in pilots under various task demands , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[27]  Kenneth W. Bauer,et al.  Improving pilot mental workload classification through feature exploitation and combination: a feasibility study , 2005, Comput. Oper. Res..

[28]  I. Parberry,et al.  Modality Specific Assessment of Video Game Player's Cognitive Workload Using Off-the-Shelf Electroencephalographic Technologies , 2014 .

[29]  Theiler,et al.  Efficient algorithm for estimating the correlation dimension from a set of discrete points. , 1987, Physical review. A, General physics.

[30]  N. Kannathal,et al.  Complex dynamics of epileptic EEG , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[31]  Maarten A. Hogervorst,et al.  Combining and comparing EEG, peripheral physiology and eye-related measures for the assessment of mental workload , 2014, Front. Neurosci..

[32]  Petros Maragos,et al.  Energy separation in signal modulations with application to speech analysis , 1993, IEEE Trans. Signal Process..

[33]  Gianluca Bontempi,et al.  On the Use of Variable Complementarity for Feature Selection in Cancer Classification , 2006, EvoWorkshops.

[34]  H. Abarbanel,et al.  Determining embedding dimension for phase-space reconstruction using a geometrical construction. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[35]  Barry H. Kantowitz,et al.  Mental Workload , 2020, Encyclopedia of Behavioral Medicine.

[36]  Valer Jurcak,et al.  10/20, 10/10, and 10/5 systems revisited: Their validity as relative head-surface-based positioning systems , 2007, NeuroImage.

[37]  Klaus-Robert Müller,et al.  Machine learning for real-time single-trial EEG-analysis: From brain–computer interfacing to mental state monitoring , 2008, Journal of Neuroscience Methods.

[38]  Nigel H. Lovell,et al.  Beyond Subjective Self-Rating: EEG Signal Classification of Cognitive Workload , 2015, IEEE Transactions on Autonomous Mental Development.

[39]  Karl Pearson F.R.S. LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[40]  Yaacob Sazali,et al.  Classification of human emotion from EEG using discrete wavelet transform , 2010 .

[41]  Samara L. Firebaugh,et al.  Cognitive stress recognition , 2013, 2013 IEEE International Instrumentation and Measurement Technology Conference (I2MTC).

[42]  Yosihito Maruyama,et al.  Measures of Multivariate Skewness and Kurtosis with Principal Components , 2007 .

[43]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[44]  Anne-Marie Brouwer,et al.  Measuring workload using a combination of electroencephalography and near infrared spectroscopy , 2012 .

[45]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Rodolphe J. Gentili,et al.  A Composite Cognitive Workload Assessment System in Pilots Under Various Task Demands Using Ensemble Learning , 2015, HCI.

[47]  H. Kantz,et al.  Nonlinear time series analysis , 1997 .

[48]  M. Murugappan,et al.  Time-Frequency Analysis of EEG Signals for Human Emotion Detection , 2008 .

[49]  Wen-Chin Li,et al.  Pilots' visual scan patterns and situation awareness in flight operations. , 2014, Aviation, space, and environmental medicine.

[50]  Glenn F. Wilson,et al.  Real-Time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks , 2003, Hum. Factors.

[51]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[52]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[53]  Jane Labadin,et al.  Feature selection based on mutual information , 2015, 2015 9th International Conference on IT in Asia (CITA).

[54]  Christophe Bourdin,et al.  Effectiveness of Physiological and Psychological Features to Estimate Helicopter Pilots' Workload: A Bayesian Network Approach , 2013, IEEE Transactions on Intelligent Transportation Systems.