Action prediction based on anticipatory brain potentials during simulated driving

OBJECTIVE The ability of an automobile to infer the driver's upcoming actions directly from neural signals could enrich the interaction of the car with its driver. Intelligent vehicles fitted with an on-board brain-computer interface able to decode the driver's intentions can use this information to improve the driving experience. In this study we investigate the neural signatures of anticipation of specific actions, namely braking and accelerating. APPROACH We investigated anticipatory slow cortical potentials in electroencephalogram recorded from 18 healthy participants in a driving simulator using a variant of the contingent negative variation (CNV) paradigm with Go and No-go conditions: count-down numbers followed by 'Start'/'Stop' cue. We report decoding performance before the action onset using a quadratic discriminant analysis classifier based on temporal features. MAIN RESULTS (i) Despite the visual and driving related cognitive distractions, we show the presence of anticipatory event related potentials locked to the stimuli onset similar to the widely reported CNV signal (with an average peak value of -8 μV at electrode Cz). (ii) We demonstrate the discrimination between cases requiring to perform an action upon imperative subsequent stimulus (Go condition, e.g. a 'Red' traffic light) versus events that do not require such action (No-go condition; e.g. a 'Yellow' light); with an average single trial classification performance of 0.83 ± 0.13 for braking and 0.79 ± 0.12 for accelerating (area under the curve). (iii) We show that the centro-medial anticipatory potentials are observed as early as 320 ± 200 ms before the action with a detection rate of 0.77 ± 0.12 in offline analysis. SIGNIFICANCE We show for the first time the feasibility of predicting the driver's intention through decoding anticipatory related potentials during simulated car driving with high recognition rates.

[1]  Ning Jiang,et al.  Enhanced Low-Latency Detection of Motor Intention From EEG for Closed-Loop Brain-Computer Interface Applications , 2014, IEEE Transactions on Biomedical Engineering.

[2]  H W van der Glas,et al.  Detection of onset and termination of muscle activity in surface electromyograms. , 1998, Journal of oral rehabilitation.

[3]  Chin-Teng Lin,et al.  Driver's cognitive state classification toward brain computer interface via using a generalized and supervised technology , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[4]  P. Vetter,et al.  Reliability and Stability of Contingent Negative Variation , 2000, Applied psychophysiology and biofeedback.

[5]  Lucian Gheorghe,et al.  Steering timing prediction in a driving simulator task , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[6]  T. Tamura,et al.  27th Annual Inter national Conference of the IEEE Engineering in Medicine and Biology Society , 2005 .

[7]  W. Walter,et al.  Contingent Negative Variation : An Electric Sign of Sensori-Motor Association and Expectancy in the Human Brain , 1964, Nature.

[8]  J. del R. Millán,et al.  Characterizing the EEG Correlates of Exploratory Behavior , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[10]  G. Pfurtscheller,et al.  Foot and hand area mu rhythms. , 1997, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[11]  D. Farina,et al.  Detection of movement intention from single-trial movement-related cortical potentials , 2011, Journal of neural engineering.

[12]  Roland Siegwart,et al.  Anticipation- and error-related EEG signals during realistic human-machine interaction: A study on visual and tactile feedback , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[13]  J. Millán,et al.  Detection of self-paced reaching movement intention from EEG signals , 2012, Front. Neuroeng..

[14]  M. Omachi,et al.  Traffic light detection with color and edge information , 2009, 2009 2nd IEEE International Conference on Computer Science and Information Technology.

[15]  Ricardo Chavarriaga,et al.  Detection of anticipatory brain potentials during car driving , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[17]  J R Wolpaw,et al.  Spatial filter selection for EEG-based communication. , 1997, Electroencephalography and clinical neurophysiology.

[18]  Dick de Waard,et al.  Classifying visuomotor workload in a driving simulator using subject specific spatial brain patterns , 2013, Front. Neurosci..

[19]  Fawzi Nashashibi,et al.  Real time visual traffic lights recognition based on Spot Light Detection and adaptive traffic lights templates , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[20]  Stefan Haufe,et al.  Detection of braking intention in diverse situations during simulated driving based on EEG feature combination , 2015, Journal of neural engineering.

[21]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[22]  Waldemar Kirsch,et al.  ERP correlates of linear hand movements: Distance dependent changes , 2010, Clinical Neurophysiology.

[23]  J. Millán,et al.  Single trial analysis of slow cortical potentials: a study on anticipation related potentials , 2013, Journal of neural engineering.

[24]  N. E. Loveless,et al.  The impact of warning signal intensity on reaction time and components of the contingent negative variation , 1975, Biological Psychology.

[25]  Stefan Haufe,et al.  EEG potentials predict upcoming emergency brakings during simulated driving , 2011, Journal of neural engineering.

[26]  Chin-Teng Lin,et al.  EEG-based cognitive state monitoring and predition by using the self-constructing neural fuzzy system , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[27]  B. Rockstroh,et al.  Slow potentials of the cerebral cortex and behavior. , 1990, Physiological reviews.

[28]  Stefan Haufe,et al.  Electrophysiology-based detection of emergency braking intention in real-world driving , 2014, Journal of neural engineering.