Estimating endogenous changes in task performance from EEG

Brain wave activity is known to correlate with decrements in behavioral performance as individuals enter states of fatigue, boredom, or low alertness.Many BCI technologies are adversely affected by these changes in user state, limiting their application and constraining their use to relatively short temporal epochs where behavioral performance is likely to be stable. Incorporating a passive BCI that detects when the user is performing poorly at a primary task, and adapts accordingly may prove to increase overall user performance. Here, we explore the potential for extending an established method to generate continuous estimates of behavioral performance from ongoing neural activity; evaluating the extended method by applying it to the original task domain, simulated driving; and generalizing the method by applying it to a BCI-relevant perceptual discrimination task. Specifically, we used EEG log power spectra and sequential forward floating selection (SFFS) to estimate endogenous changes in behavior in both a simulated driving task and a perceptual discrimination task. For the driving task the average correlation coefficient between the actual and estimated lane deviation was 0.37 ± 0.22 (μ ± σ). For the perceptual discrimination task we generated estimates of accuracy, reaction time, and button press duration for each participant. The correlation coefficients between the actual and estimated behavior were similar for these three metrics (accuracy = 0.25 ± 0.37, reaction time = 0.33 ± 0.23, button press duration = 0.36 ± 0.30). These findings illustrate the potential for modeling time-on-task decrements in performance from concurrent measures of neural activity.

[1]  P. Sajda,et al.  Cortically coupled computer vision for rapid image search , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Venkatesh Balasubramanian,et al.  EEG based analysis of cognitive fatigue during simulated driving , 2011 .

[3]  J. Schoffelen,et al.  Prestimulus Oscillatory Activity in the Alpha Band Predicts Visual Discrimination Ability , 2008, The Journal of Neuroscience.

[4]  Andrzej Cichocki,et al.  Whether generic model works for rapid ERP-based BCI calibration , 2013, Journal of Neuroscience Methods.

[5]  H. Gr Compensatory control in the regulation of human performance under stress and high workload; a cognitive-energetical framework. , 1997 .

[6]  Paul Sajda,et al.  Comparing Neural Correlates of Visual Target Detection in Serial Visual Presentations Having Different Temporal Correlations , 2009, Front. Hum. Neurosci..

[7]  J. Roerdink,et al.  The influence of mental fatigue and motivation on neural network dynamics; an EEG coherence study , 2009, Brain Research.

[8]  Chin-Teng Lin,et al.  Generalized EEG-Based Drowsiness Prediction System by Using a Self-Organizing Neural Fuzzy System , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[9]  Scott E. Kerick,et al.  Brain–Computer Interface Technologies in the Coming Decades , 2012, Proceedings of the IEEE.

[10]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[11]  Diane M. Beck,et al.  Pulsed Out of Awareness: EEG Alpha Oscillations Represent a Pulsed-Inhibition of Ongoing Cortical Processing , 2011, Front. Psychology.

[12]  Á. Pascual-Leone,et al.  Spontaneous fluctuations in posterior alpha-band EEG activity reflect variability in excitability of human visual areas. , 2008, Cerebral cortex.

[13]  K. Müller,et al.  Finding stationary subspaces in multivariate time series. , 2009, Physical review letters.

[14]  Tong Heng Lee,et al.  Dynamically weighted ensemble classification for non-stationary EEG processing , 2013, Journal of neural engineering.

[15]  Paula A. Desmond,et al.  Personality and multiple dimensions of task-induced fatigue: a study of simulated driving , 1998 .

[16]  Evangelos Bekiaris,et al.  Using EEG spectral components to assess algorithms for detecting fatigue , 2009, Expert Syst. Appl..

[17]  Jon Touryan,et al.  Integration of Automated Neural Processing into an Army-Relevant Multitasking Simulation Environment , 2013, HCI.

[18]  Cuntai Guan,et al.  Unsupervised Brain Computer Interface Based on Intersubject Information and Online Adaptation , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Timothy H. Monk,et al.  A visual analogue scale technique to measure global vigor and affect , 1989, Psychiatry Research.

[20]  Tzyy-Ping Jung,et al.  Adaptive EEG-Based Alertness Estimation System by Using ICA-Based Fuzzy Neural Networks , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[21]  O. Jensen,et al.  Shaping Functional Architecture by Oscillatory Alpha Activity: Gating by Inhibition , 2010, Front. Hum. Neurosci..

[22]  Shih-Fu Chang,et al.  Cortically-Coupled Computer Vision , 2010, Brain-Computer Interfaces.

[23]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[24]  Yuan-Pin Lin,et al.  Co-modulatory spectral changes in independent brain processes are correlated with task performance , 2012, NeuroImage.

[25]  Peter-Marc Fortune,et al.  Fatigue and Stress , 2013 .

[26]  Gwen Littlewort,et al.  Machine Learning Systems for Detecting Driver Drowsiness , 2009 .

[27]  L Tonin,et al.  An online EEG BCI based on covert visuospatial attention in absence of exogenous stimulation , 2013, Journal of neural engineering.

[28]  Lucas C. Parra,et al.  Cortical origins of response time variability during rapid discrimination of visual objects , 2005, NeuroImage.

[29]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[30]  Tzyy-Ping Jung,et al.  EEG-based drowsiness estimation for safety driving using independent component analysis , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[31]  T. Sejnowski,et al.  Estimating alertness from the EEG power spectrum , 1997, IEEE Transactions on Biomedical Engineering.

[32]  James S. P. Macdonald,et al.  Trial-by-Trial Variations in Subjective Attentional State are Reflected in Ongoing Prestimulus EEG Alpha Oscillations , 2011, Front. Psychology.

[33]  José del R. Millán,et al.  Improving Human Performance in a Real Operating Environment through Real-Time Mental Workload Detection , 2007 .

[34]  A. Craig,et al.  Reproducibility of the spectral components of the electroencephalogram during driver fatigue. , 2005, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[35]  T O Zander,et al.  Context-aware brain–computer interfaces: exploring the information space of user, technical system and environment , 2012, Journal of neural engineering.

[36]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[37]  Dongrui Wu,et al.  Collaborative Filtering for Brain-Computer Interaction Using Transfer Learning and Active Class Selection , 2013, PloS one.

[38]  Simon Hanslmayr,et al.  Prestimulus oscillations predict visual perception performance between and within subjects , 2007, NeuroImage.

[39]  Robert Oostenveld,et al.  Estimating workload using EEG spectral power and ERPs in the n-back task , 2012, Journal of neural engineering.

[40]  Tzyy-Ping Jung,et al.  EURASIP Journal on Applied Signal Processing 2005:19, 3165–3174 c ○ 2005 Hindawi Publishing Corporation Estimating Driving Performance Based on EEG Spectrum Analysis , 2005 .

[41]  Richard D. Jones,et al.  EEG-Based Lapse Detection With High Temporal Resolution , 2007, IEEE Transactions on Biomedical Engineering.

[42]  P. Maruff,et al.  Acoustic analysis of the effects of sustained wakefulness on speech. , 2010, The Journal of the Acoustical Society of America.

[43]  Diane M. Beck,et al.  To See or Not to See: Prestimulus α Phase Predicts Visual Awareness , 2009, The Journal of Neuroscience.

[44]  R. Norton,et al.  Driver sleepiness and risk of serious injury to car occupants: population based case control study , 2002, BMJ : British Medical Journal.

[45]  Adrian J. Ecker,et al.  Effects of night work, sleep loss and time on task on simulated threat detection performance. , 2008, Sleep.

[46]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[47]  M. Jeng,et al.  Driver fatigue and highway driving: A simulator study , 2008, Physiology & Behavior.

[48]  J. Touryan,et al.  Real-Time Measurement of Face Recognition in Rapid Serial Visual Presentation , 2011, Front. Psychology.

[49]  Wojciech Samek,et al.  Transferring Subspaces Between Subjects in Brain--Computer Interfacing , 2012, IEEE Transactions on Biomedical Engineering.

[50]  A. Appels,et al.  Fatigue and Stress , 2007 .

[51]  Yijun Wang,et al.  Visual and Auditory Brain–Computer Interfaces , 2014, IEEE Transactions on Biomedical Engineering.

[52]  Á. Pascual-Leone,et al.  α-Band Electroencephalographic Activity over Occipital Cortex Indexes Visuospatial Attention Bias and Predicts Visual Target Detection , 2006, The Journal of Neuroscience.

[53]  John H. L. Hansen,et al.  In-Vehicle Corpus and Signal Processing for Driver Behavior , 2008 .

[54]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[55]  T. Åkerstedt,et al.  Subjective and objective sleepiness in the active individual. , 1990, The International journal of neuroscience.

[56]  G Wotzka,et al.  Fatigue and stress in air traffic controllers. , 1971, Ergonomics.

[57]  John J. Foxe,et al.  The Role of Alpha-Band Brain Oscillations as a Sensory Suppression Mechanism during Selective Attention , 2011, Front. Psychology.

[58]  Chris Berka,et al.  Eeg-Derived Estimators of Present and Future Cognitive Performance , 2011, Front. Hum. Neurosci..

[59]  Tzyy-Ping Jung,et al.  Tonic and phasic EEG and behavioral changes induced by arousing feedback , 2010, NeuroImage.

[60]  Anna Anund,et al.  Detecting Driver Sleepiness Using Optimized Nonlinear Combinations of Sleepiness Indicators , 2011, IEEE Transactions on Intelligent Transportation Systems.

[61]  G. R. J. Hockey Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework , 1997, Biological Psychology.

[62]  Marcia Grabowecky,et al.  Attention induces synchronization-based response gain in steady-state visual evoked potentials , 2007, Nature Neuroscience.

[63]  Ricardo Chavarriaga,et al.  Context-Aware Brain-Computer Interfaces , 2010 .

[64]  K Yu,et al.  Bilinear common spatial pattern for single-trial ERP-based rapid serial visual presentation triage. , 2012, Journal of neural engineering.

[65]  Chin-Teng Lin,et al.  A Real-Time Wireless Brain–Computer Interface System for Drowsiness Detection , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[66]  Translation of EEG-Based Performance Prediction Models to Rapid Serial Visual Presentation Tasks , 2013, HCI.

[67]  G. Borghini,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[68]  Gregory Apker,et al.  Combined Linear Regression and Quadratic Classification Approach for an EEG-Based Prediction of Driver Performance , 2013, HCI.

[69]  Claus Bahlmann,et al.  In a Blink of an Eye and a Switch of a Transistor: Cortically Coupled Computer Vision , 2010, Proceedings of the IEEE.

[70]  M. Golz,et al.  Evaluation of PERCLOS based current fatigue monitoring technologies , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.