Masked and unmasked error-related potentials during continuous control and feedback

The detection of error-related potentials (ErrPs) in tasks with discrete feedback is well established in the brain-computer interface (BCI) field. However, the decoding of ErrPs in tasks with continuous feedback is still in its early stages. OBJECTIVE We developed a task in which subjects have continuous control of a cursor's position by means of a joystick. The cursor's position was shown to the participants in two different modalities of continuous feedback: normal and jittered. The jittered feedback was created to mimic the instability that could exist if participants controlled the trajectory directly with brain signals. APPROACH This paper studies the electroencephalographic (EEG)-measurable signatures caused by a loss of control over the cursor's trajectory, causing a target miss. MAIN RESULTS In both feedback modalities, time-locked potentials revealed the typical frontal-central components of error-related potentials. Errors occurring during the jittered feedback (masked errors) were delayed in comparison to errors occurring during normal feedback (unmasked errors). Masked errors displayed lower peak amplitudes than unmasked errors. Time-locked classification analysis allowed a good distinction between correct and error classes (average Cohen-[Formula: see text], average TPR  =  81.8% and average TNR  =  96.4%). Time-locked classification analysis between masked error and unmasked error classes revealed results at chance level (average Cohen-[Formula: see text], average TPR  =  60.9% and average TNR  =  58.3%). Afterwards, we performed asynchronous detection of ErrPs, combining both masked and unmasked trials. The asynchronous detection of ErrPs in a simulated online scenario resulted in an average TNR of 84.0% and in an average TPR of 64.9%. SIGNIFICANCE The time-locked classification results suggest that the masked and unmasked errors were indistinguishable in terms of classification. The asynchronous classification results suggest that the feedback modality did not hinder the asynchronous detection of ErrPs.

[1]  Vicenç Gómez,et al.  Adaptive Classification on Brain-Computer Interfaces Using Reinforcement Signals , 2012, Neural Computation.

[2]  A. Buttfield,et al.  Towards a robust BCI: error potentials and online learning , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  M. McHugh.  Interrater reliability: the kappa statistic , 2012, Biochemia medica.

[4]  N. Yeung,et al.  Decision Processes in Human Performance Monitoring , 2010, The Journal of Neuroscience.

[5]  Gernot R. Müller-Putz,et al.  Single Versus Multiple Events Error Potential Detection in a BCI-Controlled Car Game With Continuous and Discrete Feedback , 2016, IEEE Transactions on Biomedical Engineering.

[6]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[7]  Luis Montesano,et al.  Single trial recognition of error-related potentials during observation of robot operation , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[8]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[9]  Andreas Schulze-Bonhage,et al.  Error-related electrocorticographic activity in humans during continuous movements , 2012, Journal of neural engineering.

[10]  Martin Spüler,et al.  Error-related potentials during continuous feedback: using EEG to detect errors of different type and severity , 2015, Front. Hum. Neurosci..

[11]  P. Sajda,et al.  Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[12]  H. Endo,et al.  Mental fatigue and impaired response processes: event-related brain potentials in a Go/NoGo task. , 2009, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  H. Bekkering,et al.  Modulation of activity in medial frontal and motor cortices during error observation , 2004, Nature Neuroscience.

[14]  Marco Steinhauser,et al.  Error awareness as evidence accumulation: effects of speed-accuracy trade-off on error signaling , 2012, Front. Hum. Neurosci..

[15]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[17]  Ricardo Chavarriaga,et al.  Errare machinale est: the use of error-related potentials in brain-machine interfaces , 2014, Front. Neurosci..

[18]  Cuntai Guan,et al.  Adaptive estimation of hand movement trajectory in an EEG based brain–computer interface system , 2015, Journal of neural engineering.

[19]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[20]  J. Hohnsbein,et al.  ERP components on reaction errors and their functional significance: a tutorial , 2000, Biological Psychology.

[21]  Gernot R. Müller-Putz,et al.  Decoding of velocities and positions of 3D arm movement from EEG , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[22]  Joseph DelPreto,et al.  Correcting robot mistakes in real time using EEG signals , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Klaus Gramann,et al.  Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activity , 2016, Proceedings of the National Academy of Sciences.

[24]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[25]  Changsong Zhou,et al.  Reconstructing ERP amplitude effects after compensating for trial-to-trial latency jitter: A solution based on a novel application of residue iteration decomposition. , 2016, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[26]  M. A. Pisauro,et al.  Neural correlates of evidence accumulation during value-based decisions revealed via simultaneous EEG-fMRI , 2017, Nature Communications.

[27]  José del R. Millán,et al.  Simultaneous Real-Time Detection of Motor Imagery and Error-Related Potentials for Improved BCI Accuracy , 2008 .

[28]  Ricardo Chavarriaga,et al.  Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control , 2015, Scientific Reports.

[29]  Ricardo Chavarriaga,et al.  Latency correction of error potentials between different experiments reduces calibration time for single-trial classification , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[30]  Elsa Andrea Kirchner,et al.  Handling Few Training Data: Classifier Transfer Between Different Types of Error-Related Potentials , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  G. Pfurtscheller,et al.  EEG-based communication: presence of an error potential , 2000, Clinical Neurophysiology.

[32]  Vicenç Gómez,et al.  On the use of interaction error potentials for adaptive brain computer interfaces , 2011, Neural Networks.

[33]  Qiong Zhang,et al.  Improving motor imagery BCI with user response to feedback , 2017 .

[34]  L. Montesano,et al.  Analysis and asynchronous detection of gradually unfolding errors during monitoring tasks , 2015, Journal of neural engineering.

[35]  Febo Cincotti,et al.  Towards Noninvasive Hybrid Brain–Computer Interfaces: Framework, Practice, Clinical Application, and Beyond , 2015, Proceedings of the IEEE.

[36]  Ricardo Chavarriaga,et al.  Asynchronous Decoding of Error Potentials during the Monitoring of a Reaching Task , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[37]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[38]  Clay B. Holroyd,et al.  The feedback-related negativity reflects the binary evaluation of good versus bad outcomes , 2006, Biological Psychology.

[39]  Andreea Ioana Sburlea,et al.  Error-related potentials with masked and unmasked onset during continuous control and feedback , 2017 .

[40]  S. Kelly,et al.  A supramodal accumulation-to-bound signal that determines perceptual decisions in humans , 2012, Nature Neuroscience.

[41]  José del R. Millán,et al.  You Are Wrong! - Automatic Detection of Interaction Errors from Brain Waves , 2005, IJCAI.

[42]  J Spinnato,et al.  Detecting single-trial EEG evoked potential using a wavelet domain linear mixed model: application to error potentials classification , 2015, Journal of neural engineering.

[43]  Christa Neuper,et al.  Error potential detection during continuous movement of an artificial arm controlled by brain–computer interface , 2012, Medical & Biological Engineering & Computing.

[44]  Clay B. Holroyd,et al.  ERP correlates of feedback and reward processing in the presence and absence of response choice. , 2005, Cerebral cortex.

[45]  Ferran Argelaguet,et al.  Influence of Error Rate on Frustration of BCI Users , 2016, AVI.

[46]  M. Nuttin,et al.  A brain-actuated wheelchair: Asynchronous and non-invasive Brain–computer interfaces for continuous control of robots , 2008, Clinical Neurophysiology.

[47]  Gernot R. Müller-Putz,et al.  No Training, same Performance!? - a Generic P300 Classifier Approach , 2017, GBCIC.

[48]  K.-R. Muller,et al.  Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.