Neural Correlates of User-initiated Motor Success and Failure – A Brain–Computer Interface Perspective

Any motor action is, by nature, potentially accompanied by human errors. In order to facilitate development of error-tailored Brain-Computer Interface (BCI) correction systems, we focused on internal, human-initiated errors, and investigated EEG correlates of user outcome successes and errors during a continuous 3D virtual tennis game against a computer player. We used a multisensory, 3D, highly immersive environment. Missing and repelling the tennis ball were considered, as 'error' (miss) and 'success' (repel). Unlike most previous studies, where the environment "encouraged" the participant to perform a mistake, here errors happened naturally, resulting from motor-perceptual-cognitive processes of incorrect estimation of the ball kinematics, and can be regarded as user internal, self-initiated errors. Results show distinct and well-defined Event-Related Potentials (ERPs), embedded in the ongoing EEG, that differ across conditions by waveforms, scalp signal distribution maps, source estimation results (sLORETA) and time-frequency patterns, establishing a series of typical features that allow valid discrimination between user internal outcome success and error. The significant delay in latency between positive peaks of error- and success-related ERPs, suggests a cross-talk between top-down and bottom-up processing, represented by an outcome recognition process, in the context of the game world. Success-related ERPs had a central scalp distribution, while error-related ERPs were centro-parietal. The unique characteristics and sharp differences between EEG correlates of error/success provide the crucial components for an improved BCI system. The features of the EEG waveform can be used to detect user action outcome, to be fed into the BCI correction system.

[1]  Clay B. Holroyd,et al.  Electroencephalographic correlates of target and outcome errors , 2008, Experimental Brain Research.

[2]  Miriam Reiner,et al.  The Effects of 3D Collocated Presentation of Visuo-haptic Information on Performance in a Complex Realistic Visuo-motor Task , 2010, EuroHaptics.

[3]  Erich Schröger,et al.  Digital filter design for electrophysiological data – a practical approach , 2015, Journal of Neuroscience Methods.

[4]  J. Hohnsbein,et al.  ERP components on reaction errors and their functional significance: a tutorial , 2000, Biological Psychology.

[5]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[6]  A Thron,et al.  Playing piano in the mind--an fMRI study on music imagery and performance in pianists. , 2004, Brain research. Cognitive brain research.

[7]  Charles P. Lin,et al.  Analyzing Structure and Function of Vascularization in Engineered Bone Tissue by Video-Rate Intravital Microscopy and 3D Image Processing. , 2015, Tissue engineering. Part C, Methods.

[8]  J. Krakauer,et al.  A computational neuroanatomy for motor control , 2008, Experimental Brain Research.

[9]  Miriam Reiner,et al.  Stroop Interference and Facilitation Effects in Kinesthetic and Haptic Tasks , 2010, Adv. Hum. Comput. Interact..

[10]  Iñaki Iturrate,et al.  Robot reinforcement learning using EEG-based reward signals , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Martin Spüler,et al.  Error-related potentials during continuous feedback: using EEG to detect errors of different type and severity , 2015, Front. Hum. Neurosci..

[12]  Daniel Memmert,et al.  Attentional Mechanisms in Sports Via Brain-Electrical Event-Related Potentials , 2009, Research quarterly for exercise and sport.

[13]  Wim Van Paesschen,et al.  Canonical Correlation Analysis Applied to Remove Muscle Artifacts From the Electroencephalogram , 2006, IEEE Transactions on Biomedical Engineering.

[14]  Hugh Garavan,et al.  Automaticity and Reestablishment of Executive ControlAn fMRI Study , 2006, Journal of Cognitive Neuroscience.

[15]  Andreas Schulze-Bonhage,et al.  Error-related electrocorticographic activity in humans during continuous movements , 2012, Journal of neural engineering.

[16]  M. Lobo,et al.  Shining light on motivation, emotion, and memory processes , 2015, Front. Behav. Neurosci..

[17]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[18]  Gerald E Loeb,et al.  Cognitive signals for brain–machine interfaces in posterior parietal cortex include continuous 3D trajectory commands , 2012, Proceedings of the National Academy of Sciences.

[19]  Benoit Bediou,et al.  Delayed monitoring of accuracy errors compared to commission errors in ACC , 2012, NeuroImage.

[20]  Yasmin L. Hashambhoy,et al.  Neural Correlates of Reach Errors , 2005, The Journal of Neuroscience.

[21]  John J. Foxe,et al.  The role of cingulate cortex in the detection of errors with and without awareness: a high‐density electrical mapping study , 2007, The European journal of neuroscience.

[22]  L. Mo,et al.  Event-related potential effects of superior action anticipation in professional badminton players , 2011, Neuroscience Letters.

[23]  Panagiotis D. Bamidis,et al.  REG-ICA: A hybrid methodology combining Blind Source Separation and regression techniques for the rejection of ocular artifacts , 2011, Biomed. Signal Process. Control..

[24]  S. Makeig Auditory event-related dynamics of the EEG spectrum and effects of exposure to tones. , 1993, Electroencephalography and clinical neurophysiology.

[25]  Wouter Hulstijn,et al.  Action monitoring in motor control: ERPs following selection and execution errors in a force production task. , 2003, Psychophysiology.

[26]  G. Wilson,et al.  Removal of ocular artifacts from electro-encephalogram by adaptive filtering , 2004, Medical and Biological Engineering and Computing.

[27]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[28]  Scott T. Grafton,et al.  Role of the posterior parietal cortex in updating reaching movements to a visual target , 1999, Nature Neuroscience.

[29]  Scott E Kerick,et al.  Independent component analysis of dynamic brain responses during visuomotor adaptation , 2004, NeuroImage.

[30]  Clay B. Holroyd,et al.  The feedback-related negativity reflects the binary evaluation of good versus bad outcomes , 2006, Biological Psychology.

[31]  S. Scott The role of primary motor cortex in goal-directed movements: insights from neurophysiological studies on non-human primates , 2003, Current Opinion in Neurobiology.

[32]  Toshiaki Imada,et al.  Theta brain rhythms index perceptual narrowing in infant speech perception , 2013, Front. Psychol..

[33]  Markus Raab,et al.  Analyzing a complex visuomotor tracking task with brain-electrical event related potentials. , 2005, Human movement science.

[34]  S. Luck An Introduction to the Event-Related Potential Technique , 2005 .

[35]  Miriam Reiner,et al.  The role of haptics in immersive telecommunication environments , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[37]  Miriam Reiner,et al.  Is Learning in Low Immersive Environments Carried over to High Immersive Environments? , 2012, Adv. Hum. Comput. Interact..

[38]  Zoubin Ghahramani,et al.  Computational principles of movement neuroscience , 2000, Nature Neuroscience.

[39]  Jean Régis,et al.  Ultra-Rapid Sensory Responses in the Human Frontal Eye Field Region , 2009, The Journal of Neuroscience.

[40]  Heiko Maurer,et al.  Neural correlates of error prediction in a complex motor task , 2015, Front. Behav. Neurosci..

[41]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[42]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[43]  Phillip Wolff,et al.  Causal reasoning with forces , 2015, Front. Hum. Neurosci..

[44]  M. Reiner,et al.  Perspectives and possible applications of the rubber hand and virtual hand illusion in non-invasive rehabilitation: Technological improvements and their consequences , 2014, Neuroscience & Biobehavioral Reviews.

[45]  Stéphane Lehéricy,et al.  Distinct prefrontal activations in processing sequence at the sentence and script level: An fMRI study , 1999, Neuropsychologia.

[46]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[47]  Cheuk Y. Tang,et al.  Thalamocortical circuits: fMRI assessment of the pulvinar and medial dorsal nucleus in normal volunteers , 2006, Neuroscience Letters.

[48]  C. Mehring,et al.  Detection of Error Related Neuronal Responses Recorded by Electrocorticography in Humans during Continuous Movements , 2013, PloS one.

[49]  M. Mustari,et al.  Signal Processing and Distribution in Cortical‐Brainstem Pathways for Smooth Pursuit Eye Movements , 2009, Annals of the New York Academy of Sciences.

[50]  Holger Hill An event-related potential evoked by movement planning is modulated by performance and learning in visuomotor control , 2009, Experimental Brain Research.

[51]  Hillel Pratt,et al.  Influence of task-related ipsilateral hand movement on motor cortex excitability , 2001, Clinical Neurophysiology.

[52]  S. Hillyard,et al.  Modulations of sensory-evoked brain potentials indicate changes in perceptual processing during visual-spatial priming. , 1991, Journal of experimental psychology. Human perception and performance.

[53]  Max E. Valentinuzzi,et al.  Artifact removal from EEG signals using adaptive filters in cascade , 2007 .

[54]  José del R. Millán,et al.  Simultaneous Real-Time Detection of Motor Imagery and Error-Related Potentials for Improved BCI Accuracy , 2008 .

[55]  R D Pascual-Marqui,et al.  Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. , 2002, Methods and findings in experimental and clinical pharmacology.

[56]  D. Mathalon,et al.  Event-related EEG time-frequency analysis: an overview of measures and an analysis of early gamma band phase locking in schizophrenia. , 2008, Schizophrenia bulletin.

[57]  Scott T. Grafton,et al.  Forward modeling allows feedback control for fast reaching movements , 2000, Trends in Cognitive Sciences.

[58]  Pablo Laguna,et al.  Bioelectrical Signal Processing in Cardiac and Neurological Applications , 2005 .

[59]  G. Pourtois,et al.  What is Bottom-Up and What is Top-Down in Predictive Coding? , 2013, Front. Psychol..