EEG-based decoding of error-related brain activity in a real-world driving task

OBJECTIVES Recent studies have started to explore the implementation of brain-computer interfaces (BCI) as part of driving assistant systems. The current study presents an EEG-based BCI that decodes error-related brain activity. Such information can be used, e.g., to predict driver's intended turning direction before reaching road intersections. APPROACH We executed experiments in a car simulator (N = 22) and a real car (N = 8). While subject was driving, a directional cue was shown before reaching an intersection, and we classified the presence or not of an error-related potentials from EEG to infer whether the cued direction coincided with the subject's intention. In this protocol, the directional cue can correspond to an estimation of the driving direction provided by a driving assistance system. We analyzed ERPs elicited during normal driving and evaluated the classification performance in both offline and online tests. RESULTS An average classification accuracy of 0.698 ± 0.065 was obtained in offline experiments in the car simulator, while tests in the real car yielded a performance of 0.682 ± 0.059. The results were significantly higher than chance level for all cases. Online experiments led to equivalent performances in both simulated and real car driving experiments. These results support the feasibility of decoding these signals to help estimating whether the driver's intention coincides with the advice provided by the driving assistant in a real car. SIGNIFICANCE The study demonstrates a BCI system in real-world driving, extending the work from previous simulated studies. As far as we know, this is the first online study in real car decoding driver's error-related brain activity. Given the encouraging results, the paradigm could be further improved by using more sophisticated machine learning approaches and possibly be combined with applications in intelligent vehicles.

[1]  K. R. Ridderinkhof,et al.  Impaired cognitive control and reduced cingulate activity during mental fatigue. , 2005, Brain research. Cognitive brain research.

[2]  Stefan Haufe,et al.  Detection of braking intention in diverse situations during simulated driving based on EEG feature combination , 2015, Journal of neural engineering.

[3]  Jose M Carmena,et al.  Invasive or noninvasive: understanding brain-machine interface technology. , 2010, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[4]  Dick de Waard,et al.  Classifying visuomotor workload in a driving simulator using subject specific spatial brain patterns , 2013, Front. Neurosci..

[5]  Yijun Wang,et al.  Phase Synchrony Measurement in Motor Cortex for Classifying Single-trial EEG during Motor Imagery , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[6]  Ricardo Chavarriaga,et al.  Action prediction based on anticipatory brain potentials during simulated driving , 2015, Journal of neural engineering.

[7]  Ricardo Chavarriaga,et al.  Errare machinale est: the use of error-related potentials in brain-machine interfaces , 2014, Front. Neurosci..

[8]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[9]  Michael Schrauf,et al.  EEG alpha spindles and prolonged brake reaction times during auditory distraction in an on-road driving study. , 2014, Accident; analysis and prevention.

[10]  Miguel A. L. Nicolelis,et al.  Brain–machine interfaces: past, present and future , 2006, Trends in Neurosciences.

[11]  W. Gehring,et al.  Neural Systems for Error Monitoring , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[12]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection , 2012, J. Mach. Learn. Res..

[13]  H. Bekkering,et al.  Modulation of activity in medial frontal and motor cortices during error observation , 2004, Nature Neuroscience.

[14]  M. Frank,et al.  Frontal theta as a mechanism for cognitive control , 2014, Trends in Cognitive Sciences.

[15]  Stefan Haufe,et al.  Electrophysiology-based detection of emergency braking intention in real-world driving , 2014, Journal of neural engineering.

[16]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[17]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[18]  Lucian Gheorghe,et al.  Inferring driver's turning direction through detection of error related brain activity , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[19]  John J. B. Allen,et al.  Theta lingua franca: a common mid-frontal substrate for action monitoring processes. , 2012, Psychophysiology.

[20]  Lucian Gheorghe,et al.  Steering timing prediction in a driving simulator task , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[21]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Iñaki Iturrate,et al.  Brain connectivity in continuous error tasks , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[23]  Clemens Brunner,et al.  Single-trial connectivity estimation for classification of motor imagery data , 2013, Journal of neural engineering.

[24]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Mutual Information Feature Selection , 2012 .

[25]  Michael Schrauf,et al.  EEG alpha spindle measures as indicators of driver fatigue under real traffic conditions , 2011, Clinical Neurophysiology.

[26]  P. Sajda,et al.  Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  Ricardo Chavarriaga,et al.  Discriminant brain connectivity patterns of performance monitoring at average and single-trial levels , 2015, NeuroImage.

[28]  Ricardo Chavarriaga,et al.  Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control , 2015, Scientific Reports.

[29]  C. Joyce,et al.  Automatic removal of eye movement and blink artifacts from EEG data using blind component separation. , 2004, Psychophysiology.

[30]  Wei-Yen Hsu,et al.  EEG-based motor imagery classification using enhanced active segment selection and adaptive classifier , 2011, Comput. Biol. Medicine.

[31]  Chin-Teng Lin,et al.  Driver's cognitive state classification toward brain computer interface via using a generalized and supervised technology , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[32]  Stefan Haufe,et al.  EEG potentials predict upcoming emergency brakings during simulated driving , 2011, Journal of neural engineering.