Factors that affect error potentials during a grasping task: toward a hybrid natural movement decoding BCI

OBJECTIVE In this manuscript, we consider factors that may affect the design of a hybrid brain-computer interface (BCI). We combine neural correlates of natural movements and interaction error-related potentials (ErrP) to perform a 3D reaching task, focusing on the impact that such factors have on the evoked ErrP signatures and in their classification. APPROACH Users attempted to control a 3D virtual interface that simulated their own hand, to reach and grasp two different objects. Three factors of interest were modulated during the experimentation: (1) execution speed of the grasping, (2) type of grasping and (3) mental strategy (motor imagery or real motion) used to produce motor commands. Thirteen healthy subjects carried out the protocol. The peaks and latencies of the ErrP were analyzed for the different factors as well as the classification performance. MAIN RESULTS ErrP are evoked for erroneous commands decoded from neural correlates of natural movements. The analysis of variance (ANOVA) analyses revealed that latency and magnitude of the most characteristic ErrP peaks were significantly influenced by the speed at which the grasping was executed, but not the type of grasp. This resulted in an greater accuracy of single-trial decoding of errors for fast movements (75.65%) compared to slow ones (68.99%). SIGNIFICANCE Understanding the effects of combining paradigms is a first step to design hybrid BCI that optimize decoding accuracy and can be deployed in motor substitution and neuro-rehabilitation applications.

[1]  Gernot R Müller-Putz,et al.  Upper limb movements can be decoded from the time-domain of low-frequency EEG , 2017, PloS one.

[2]  Michael X. Cohen,et al.  Reward expectation modulates feedback-related negativity and EEG spectra , 2007, NeuroImage.

[3]  Elsa Andrea Kirchner,et al.  Handling Few Training Data: Classifier Transfer Between Different Types of Error-Related Potentials , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  J. Polich,et al.  EEG and ERP assessment of normal aging. , 1997, Electroencephalography and clinical neurophysiology.

[5]  Andreea Ioana Sburlea,et al.  Towards non-invasive EEG-based arm/hand-control in users with spinal cord injury , 2017, 2017 5th International Winter Conference on Brain-Computer Interface (BCI).

[6]  Floris P. de Lange,et al.  How Prediction Errors Shape Perception, Attention, and Motivation , 2012, Front. Psychology.

[7]  Gernot R. Müller-Putz,et al.  A Single-Switch BCI Based on Passive and imagined movements: toward Restoring Communication in Minimally Conscious patients , 2013, Int. J. Neural Syst..

[8]  Á. Gil-Agudo,et al.  Continuous decoding of movement intention of upper limb self-initiated analytic movements from pre-movement EEG correlates , 2014, Journal of NeuroEngineering and Rehabilitation.

[9]  Robert W. McCarley,et al.  A Topographic Study of ERPs Elicited by Visual Feature Discrimination , 2004, Brain Topography.

[10]  Salvatore Maria Aglioti,et al.  Wronger than wrong: Graded mapping of the errors of an avatar in the performance monitoring system of the onlooker , 2018, NeuroImage.

[11]  Andreas Schulze-Bonhage,et al.  Decoding natural grasp types from human ECoG , 2012, NeuroImage.

[12]  J. del R. Millán,et al.  Multimodal Fusion of Muscle and Brain Signals for a Hybrid-BCI , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[13]  W. Klimesch EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis , 1999, Brain Research Reviews.

[14]  Stephen Kwek,et al.  Applying Support Vector Machines to Imbalanced Datasets , 2004, ECML.

[15]  Vera Kaiser,et al.  Hybrid brain-computer interfaces and hybrid neuroprostheses for restoration of upper limb functions in individuals with high-level spinal cord injury , 2013, Artif. Intell. Medicine.

[16]  Michael X. Cohen,et al.  Error-related medial frontal theta activity predicts cingulate-related structural connectivity , 2011, NeuroImage.

[17]  Miriam Zacksenhouse,et al.  Distinct electroencephalographic responses to disturbances and distractors during continuous reaching movements , 2016, Brain Research.

[18]  Salvatore Maria Aglioti,et al.  Embodying Others in Immersive Virtual Reality: Electro-Cortical Signatures of Monitoring the Errors in the Actions of an Avatar Seen from a First-Person Perspective , 2016, The Journal of Neuroscience.

[19]  Hasan Onur Keles,et al.  Hybrid EEG-fNIRS Asynchronous Brain-Computer Interface for Multiple Motor Tasks , 2016, PloS one.

[20]  G Pfurtscheller,et al.  Self-initiation of EEG-based brain-computer communication using the heart rate response. , 2007, Journal of neural engineering.

[21]  Xavier Artusi,et al.  Performance of a Simulated Adaptive BCI Based on Experimental Classification of Movement-Related and Error Potentials , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[22]  S. Kelly,et al.  The classic P300 encodes a build‐to‐threshold decision variable , 2015, The European journal of neuroscience.

[23]  Gernot R. Müller-Putz,et al.  Functional Rehabilitation of the Paralyzed Upper Extremity After Spinal Cord Injury by Noninvasive Hybrid Neuroprostheses , 2015, Proceedings of the IEEE.

[24]  R Chavarriaga,et al.  Latency correction of event-related potentials between different experimental protocols. , 2014, Journal of neural engineering.

[25]  Anatole Lécuyer,et al.  Combining BCI with Virtual Reality: Towards New Applications and Improved BCI , 2012 .

[26]  G. Pfurtscheller,et al.  A fully automated correction method of EOG artifacts in EEG recordings , 2007, Clinical Neurophysiology.

[27]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[28]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[30]  Luca T. Mainardi,et al.  Online Detection of P300 and Error Potentials in a BCI Speller , 2010, Comput. Intell. Neurosci..

[31]  Ricardo Chavarriaga,et al.  Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control , 2015, Scientific Reports.

[32]  Mickaël Causse,et al.  EEG/ERP as a Measure of Mental Workload in a Simple Piloting Task , 2015 .

[33]  Vicenç Gómez,et al.  On the use of interaction error potentials for adaptive brain computer interfaces , 2011, Neural Networks.

[34]  Wolfgang Rosenstiel,et al.  Online use of error-related potentials in healthy users and people with severe motor impairment increases performance of a P300-BCI , 2012, Clinical Neurophysiology.

[35]  G. Pfurtscheller,et al.  EEG-based communication: presence of an error potential , 2000, Clinical Neurophysiology.

[36]  Michela Balconi,et al.  Dorsolateral pFC and the Representation of the Incorrect Use of an Object: The Transcranial Direct Current Stimulation Effect on N400 for Visual and Linguistic Stimuli , 2014, Journal of Cognitive Neuroscience.

[37]  Andrés Úbeda,et al.  Error Potential Detection to Assist Movement Intention Decoding in Stroke Patients , 2013 .

[38]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[39]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[40]  L. Cohen,et al.  Brain–machine interface in chronic stroke rehabilitation: A controlled study , 2013, Annals of neurology.

[41]  S. Segalowitz,et al.  Error detection in patients with lesions to the medial prefrontal cortex: an ERP study , 2004, Neuropsychologia.

[42]  M. Frank,et al.  Frontal theta as a mechanism for cognitive control , 2014, Trends in Cognitive Sciences.

[43]  John J. B. Allen,et al.  Prelude to and Resolution of an Error: EEG Phase Synchrony Reveals Cognitive Control Dynamics during Action Monitoring , 2009, The Journal of Neuroscience.

[44]  Paul E. Summers,et al.  Preservation of motor programs in paraplegics as demonstrated by attempted and imagined foot movements , 2008, NeuroImage.

[45]  L. Montesano,et al.  Analysis and asynchronous detection of gradually unfolding errors during monitoring tasks , 2015, Journal of neural engineering.

[46]  J. Hohnsbein,et al.  Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. , 1991, Electroencephalography and clinical neurophysiology.

[47]  Christa Neuper,et al.  Error potential detection during continuous movement of an artificial arm controlled by brain–computer interface , 2012, Medical & Biological Engineering & Computing.

[48]  Vera Kaiser,et al.  Switching between Manual Control and Brain-Computer Interface Using Long Term and Short Term Quality Measures , 2011, Front. Neurosci..

[49]  Guillaume Gibert,et al.  xDAWN Algorithm to Enhance Evoked Potentials: Application to Brain–Computer Interface , 2009, IEEE Transactions on Biomedical Engineering.

[50]  Elisabeth J. Ploran,et al.  Evidence Accumulation and the Moment of Recognition: Dissociating Perceptual Recognition Processes Using fMRI , 2007, The Journal of Neuroscience.

[51]  Chang S. Nam,et al.  Effects of Luminosity Contrast and Stimulus Duration on User Performance and Preference in a P300-Based Brain–Computer Interface , 2014, Int. J. Hum. Comput. Interact..

[52]  R D Pascual-Marqui,et al.  Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. , 2002, Methods and findings in experimental and clinical pharmacology.

[53]  Gernot R. Müller-Putz,et al.  Short time sports exercise boosts motor imagery patterns: implications of mental practice in rehabilitation programs , 2014, Front. Hum. Neurosci..

[54]  Martin Spüler,et al.  Error-related potentials during continuous feedback: using EEG to detect errors of different type and severity , 2015, Front. Hum. Neurosci..

[55]  Bernhard Graimann,et al.  Quantification and visualization of event-related changes in oscillatory brain activity in the time-frequency domain. , 2006, Progress in brain research.

[56]  Ricardo Chavarriaga,et al.  Errare machinale est: the use of error-related potentials in brain-machine interfaces , 2014, Front. Neurosci..

[57]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[58]  A. Gevins,et al.  Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style. , 2000, Cerebral cortex.

[59]  Clay B. Holroyd,et al.  Reinforcement-related brain potentials from medial frontal cortex: origins and functional significance , 2004, Neuroscience & Biobehavioral Reviews.

[60]  G R Müller-Putz,et al.  From classic motor imagery to complex movement intention decoding: The noninvasive Graz-BCI approach. , 2016, Progress in brain research.

[61]  Iñaki Iturrate,et al.  EEG single-trial classification of visual, auditive and vibratory feedback potentials in Brain-Computer Interfaces , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[62]  N. Yeung,et al.  On the ERN and the significance of errors. , 2005, Psychophysiology.

[63]  G Pfurtscheller,et al.  Toward a hybrid brain–computer interface based on imagined movement and visual attention , 2010, Journal of neural engineering.

[64]  J. Hohnsbein,et al.  ERP components on reaction errors and their functional significance: a tutorial , 2000, Biological Psychology.

[65]  Luis Montesano,et al.  Moregrasp: restoration of upper limb function in Individuals with High spinal cord injury by Multimodal Neuroprostheses for Interaction in Daily Activities , 2017, GBCIC.

[66]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[67]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[68]  J. Polich Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[69]  Febo Cincotti,et al.  Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond , 2015, Proceedings of the IEEE.

[70]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[71]  José Luis Contreras-Vidal,et al.  Decoding center-out hand velocity from MEG signals during visuomotor adaptation , 2009, NeuroImage.

[72]  Ricardo Chavarriaga,et al.  Heading for new shores! Overcoming pitfalls in BCI design. , 2017, Brain computer interfaces.