EEG coupling features: Towards mental workload measurement based on wearables

Automated mental workload measurement is particularly important in safety-critical settings, such as in nuclear plants, aviation, air traffic control, shipping, and transportation, to name a few. As an example, recent statistics have suggested that 90% of the accidents in the transport industry are due to human factors. In this paper, we explore the potential of off-the-shelf wearable technologies in monitoring mental workload in real-time, thus potentially reducing the number of accidents due to human errors. Wearable technologies, while providing the user with ease-of-use, comfort, and portability, have several limitations, such as lower quality signal readings (e.g., due to dry electrodes) and smaller number of recording sites. Such limitations place a burden on the accuracy of existing mental workload models. To overcome this limitation, we propose the use of phase-amplitude and amplitude-amplitude coupling features computed from a portable commercial electroencephalography (EEG) device. Experiments with three different tasks, namely N-back, mental rotation and visual search, show the proposed features being significantly correlated with multiple dimensions of the widely-used NASA task load index test and providing complementary information to other conventional features.

[1]  G. Orban,et al.  Attention Mechanisms in Visual SearchAn fMRI Study , 2000, Journal of Cognitive Neuroscience.

[2]  R. Knight,et al.  Shifts in Gamma Phase–Amplitude Coupling Frequency from Theta to Alpha Over Posterior Cortex During Visual Tasks , 2010, Front. Hum. Neurosci..

[3]  Michael X. Cohen,et al.  Oscillatory Activity and Phase–Amplitude Coupling in the Human Medial Frontal Cortex during Decision Making , 2009, Journal of Cognitive Neuroscience.

[4]  Lauren Reinerman-Jones,et al.  Psychophysiological Metrics for Workload are Demand-Sensitive but Multifactorial , 2014 .

[5]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[6]  H. Eichenbaum,et al.  Measuring phase-amplitude coupling between neuronal oscillations of different frequencies. , 2010, Journal of neurophysiology.

[7]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[8]  Scott Makeig,et al.  Eye Activity Correlates of Workload during a Visuospatial Memory Task , 2001, Hum. Factors.

[9]  C. D. Frith,et al.  Filtering of Distractors during Visual Search Studied by Positron Emission Tomography , 2002, NeuroImage.

[10]  O. Jensen,et al.  Cross-frequency coupling between neuronal oscillations , 2007, Trends in Cognitive Sciences.

[11]  Louise Venables,et al.  The influence of task demand and learning on the psychophysiological response. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[12]  Tiago H. Falk,et al.  MuLES: An Open Source EEG Acquisition and Streaming Server for Quick and Simple Prototyping and Recording , 2015, IUI Companion.

[13]  Daniel J. Barber,et al.  The Psychometrics of Mental Workload , 2015, Hum. Factors.

[14]  T. Falk,et al.  The effects of automated artifact removal algorithms on electroencephalography-based Alzheimer's disease diagnosis , 2014, Front. Aging Neurosci..

[15]  William B. Rouse,et al.  Modeling the dynamics of mental workload and human performance in complex systems , 1993, IEEE Trans. Syst. Man Cybern..

[16]  H. Heinze,et al.  Cortical Activations during the Mental Rotation of Different Visual Objects , 2001, NeuroImage.

[17]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[18]  J Altimiras,et al.  Understanding autonomic sympathovagal balance from short-term heart rate variations. Are we analyzing noise? , 1999, Comparative biochemistry and physiology. Part A, Molecular & integrative physiology.

[19]  P. A. Hancock,et al.  Experimental Evaluations of a Model of Mental Workload , 1989, Human factors.

[20]  Mirka Pesonen,et al.  Brain oscillatory 4–30 Hz responses during a visual n-back memory task with varying memory load , 2007, Brain Research.

[21]  Jeffrey D Schall,et al.  On the role of frontal eye field in guiding attention and saccades , 2004, Vision Research.

[22]  S. Luck,et al.  Neural sources of focused attention in visual search. , 2000, Cerebral cortex.

[23]  V. A. Makarov,et al.  Recovering EEG brain signals: Artifact suppression with wavelet enhanced independent component analysis , 2006, Journal of Neuroscience Methods.

[24]  Daniel McDuff,et al.  Remote measurement of cognitive stress via heart rate variability , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  Tanja Schultz,et al.  Mental workload during n-back task—quantified in the prefrontal cortex using fNIRS , 2014, Front. Hum. Neurosci..

[26]  Tiago H. Falk,et al.  Mutual information between inter-hemispheric EEG spectro-temporal patterns: A new feature for automated affect recognition , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[27]  P. Skudlarski,et al.  Brain Connectivity Related to Working Memory Performance , 2006, The Journal of Neuroscience.

[28]  Martin Wolf,et al.  An Efficient Algorithm for Automatic Peak Detection in Noisy Periodic and Quasi-Periodic Signals , 2012, Algorithms.

[29]  K. Miller,et al.  Exaggerated phase–amplitude coupling in the primary motor cortex in Parkinson disease , 2013, Proceedings of the National Academy of Sciences.

[30]  Sarah H. Creem,et al.  Defining the cortical visual systems: "what", "where", and "how". , 2001, Acta psychologica.

[31]  Olivier Gagnon,et al.  A Systematic Assessment of Operational Metrics for Modeling Operator Functional State , 2016, PhyCS.