Errare machinale est: the use of error-related potentials in brain-machine interfaces

The ability to recognize errors is crucial for efficient behavior. Numerous studies have identified electrophysiological correlates of error recognition in the human brain (error-related potentials, ErrPs). Consequently, it has been proposed to use these signals to improve human-computer interaction (HCI) or brain-machine interfacing (BMI). Here, we present a review of over a decade of developments toward this goal. This body of work provides consistent evidence that ErrPs can be successfully detected on a single-trial basis, and that they can be effectively used in both HCI and BMI applications. We first describe the ErrP phenomenon and follow up with an analysis of different strategies to increase the robustness of a system by incorporating single-trial ErrP recognition, either by correcting the machine's actions or by providing means for its error-based adaptation. These approaches can be applied both when the user employs traditional HCI input devices or in combination with another BMI channel. Finally, we discuss the current challenges that have to be overcome in order to fully integrate ErrPs into practical applications. This includes, in particular, the characterization of such signals during real(istic) applications, as well as the possibility of extracting richer information from them, going beyond the time-locked decoding that dominates current approaches.

[1]  P. Rabbitt Errors and error correction in choice-response tasks. , 1966, Journal of experimental psychology.

[2]  D. Koshland Frontiers in neuroscience. , 1988, Science.

[3]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[4]  J. Polich Probability and inter-stimulus interval effects on the P300 from auditory stimuli. , 1990, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[5]  J. Hohnsbein,et al.  Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks. , 1991, Electroencephalography and clinical neurophysiology.

[6]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[7]  J. Polich,et al.  P300 and probability: comparison of oddball and single-stimulus paradigms. , 1997, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[8]  G. Pfurtscheller,et al.  EEG-based communication: presence of an error potential , 2000, Clinical Neurophysiology.

[9]  J. Hohnsbein,et al.  ERP components on reaction errors and their functional significance: a tutorial , 2000, Biological Psychology.

[10]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[11]  A. Haghighat,et al.  Performance of A 3 MCNP™ for Calculation of 3-D Neutron Flux Distribution in a BWR Core Shroud , 2001 .

[12]  D. V. Cramon,et al.  Subprocesses of Performance Monitoring: A Dissociation of Error Processing and Response Competition Revealed by Event-Related fMRI and ERPs , 2001, NeuroImage.

[13]  P. Jurák,et al.  Error processing – evidence from intracerebral ERP recordings , 2002, Experimental Brain Research.

[14]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[15]  R D Pascual-Marqui,et al.  Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. , 2002, Methods and findings in experimental and clinical pharmacology.

[16]  C. Carter,et al.  The anterior cingulate as a conflict monitor: fMRI and ERP studies , 2002, Physiology & Behavior.

[17]  K.-R. Muller,et al.  Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  P. Sajda,et al.  Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[19]  Clay B. Holroyd,et al.  Errors in reward prediction are re£ected in the event-related brain potential , 2003 .

[20]  M. Herrmann,et al.  Source localization (LORETA) of the error-related-negativity (ERN/Ne) and positivity (Pe). , 2004, Brain research. Cognitive brain research.

[21]  M. Bates,et al.  The d2 Test of Attention: Construct validity and extensions in scoring techniques , 2004, Journal of the International Neuropsychological Society.

[22]  H. Bekkering,et al.  Modulation of activity in medial frontal and motor cortices during error observation , 2004, Nature Neuroscience.

[23]  Katarzyna J. Blinowska,et al.  A new method of the description of the information flow in the brain structures , 1991, Biological Cybernetics.

[24]  Clay B. Holroyd,et al.  Sensitivity of electrophysiological activity from medial frontal cortex to utilitarian and performance feedback. , 2004, Cerebral cortex.

[25]  S. Segalowitz,et al.  Development of Error‐Monitoring Event‐Related Potentials in Adolescents , 2004, Annals of the New York Academy of Sciences.

[26]  A. Engel,et al.  Trial-by-Trial Coupling of Concurrent Electroencephalogram and Functional Magnetic Resonance Imaging Identifies the Dynamics of Performance Monitoring , 2005, The Journal of Neuroscience.

[27]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[28]  Tony Scott,et al.  Analysis of performance , 2005 .

[29]  N. Yeung,et al.  On the ERN and the significance of errors. , 2005, Psychophysiology.

[30]  Clay B. Holroyd,et al.  ERP correlates of feedback and reward processing in the presence and absence of response choice. , 2005, Cerebral cortex.

[31]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[32]  P. Sajda,et al.  Cortically coupled computer vision for rapid image search , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[33]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[34]  José del R. Millán Brain-computer interaction , 2006, UIST.

[35]  W. Gehring,et al.  Neural Systems for Error Monitoring , 2007, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[36]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces , 2007, Journal of neural engineering.

[37]  John J. B. Allen,et al.  Theta EEG dynamics of the error-related negativity , 2007, Clinical Neurophysiology.

[38]  Michael X. Cohen,et al.  Reward expectation modulates feedback-related negativity and EEG spectra , 2007, NeuroImage.

[39]  J. Blumberg,et al.  Adaptive Classification for Brain Computer Interfaces , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[40]  Rabab K Ward,et al.  A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals , 2007, Journal of neural engineering.

[41]  J. Meere,et al.  Developmental changes in error monitoring: An event-related potential study , 2007, Neuropsychologia.

[42]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[43]  José del R. Millán,et al.  Simultaneous Real-Time Detection of Motor Imagery and Error-Related Potentials for Improved BCI Accuracy , 2008 .

[44]  G. Hajcak,et al.  The error-related negativity (ERN) and psychopathology: toward an endophenotype. , 2008, Clinical psychology review.

[45]  K. R. Ridderinkhof,et al.  Medial frontal cortex and response conflict: Evidence from human intracranial EEG and medial frontal cortex lesion , 2008, Brain Research.

[46]  Robert Oostenveld,et al.  Motor-cortical beta oscillations are modulated by correctness of observed action , 2008, NeuroImage.

[47]  Ricardo Chavarriaga,et al.  EEG error-related potentials detection with a Bayesian filter , 2009, 2009 4th International IEEE/EMBS Conference on Neural Engineering.

[48]  G. Hajcak,et al.  Reliability of error-related brain activity , 2009, Brain Research.

[49]  Klas Ihme,et al.  Error-related EEG patterns during tactile human-machine interaction , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[50]  Christa Neuper,et al.  Implementation of Error Detection into the Graz-Brain-Computer Interface, the Interaction Error Potential , 2009 .

[51]  Tobias Seidl,et al.  Validation of brain-machine interfaces during parabolic flight. , 2009, International review of neurobiology.

[52]  John J. B. Allen,et al.  Prelude to and Resolution of an Error: EEG Phase Synchrony Reveals Cognitive Control Dynamics during Action Monitoring , 2009, The Journal of Neuroscience.

[53]  Roland Siegwart,et al.  Article in Press Robotics and Autonomous Systems ( ) – Robotics and Autonomous Systems Brain-coupled Interaction for Semi-autonomous Navigation of an Assistive Robot , 2022 .

[54]  Luca T. Mainardi,et al.  Online Detection of P300 and Error Potentials in a BCI Speller , 2010, Comput. Intell. Neurosci..

[55]  Ricardo Chavarriaga,et al.  Adaptation of hybrid human-computer interaction systems using EEG error-related potentials , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[56]  Jordi Bieger,et al.  Towards error-free interaction , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[57]  Shih-Fu Chang,et al.  Cortically-Coupled Computer Vision , 2010, Brain-Computer Interfaces.

[58]  Ryan K. Jessup,et al.  Error Effects in Anterior Cingulate Cortex Reverse when Error Likelihood Is High , 2010, The Journal of Neuroscience.

[59]  Toyoaki Nishida From observation to interaction , 2010, SSPW '10.

[60]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[61]  A. Kübler,et al.  Motivation modulates the P300 amplitude during brain–computer interface use , 2010, Clinical Neurophysiology.

[62]  M. Matteucci,et al.  The Utility Metric: A Novel Method to Assess the Overall Performance of Discrete Brain–Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[63]  Tomohiro Yoshikawa,et al.  Reliability-Based Automatic Repeat reQuest with Error Potential-Based Error Correction for Improving P300 Speller Performance , 2010, ICONIP.

[64]  Iñaki Iturrate,et al.  Robot reinforcement learning using EEG-based reward signals , 2010, 2010 IEEE International Conference on Robotics and Automation.

[65]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[66]  Javier Minguez,et al.  Real-time recognition of feedback error-related potentials during a time-estimation task , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[67]  B. Blankertz,et al.  A New Auditory Multi-Class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue , 2010, PloS one.

[68]  Ricardo Chavarriaga,et al.  Minimizing calibration time using inter-subject information of single-trial recognition of error potentials in brain-computer interfaces , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[69]  Christa Neuper,et al.  Error potential detection during continuous movement of an artificial arm controlled by brain–computer interface , 2012, Medical & Biological Engineering & Computing.

[70]  M S Treder,et al.  Gaze-independent brain–computer interfaces based on covert attention and feature attention , 2011, Journal of neural engineering.

[71]  Vicenç Gómez,et al.  On the use of interaction error potentials for adaptive brain computer interfaces , 2011, Neural Networks.

[72]  W. Art Chaovalitwongse,et al.  Early Detection of Numerical Typing Errors Using Data Mining Techniques , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[73]  Ronald Phlypo,et al.  A non-orthogonal SVD-based decomposition for phase invariant error-related potential estimation , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[74]  Iñaki Iturrate,et al.  EEG single-trial classification of visual, auditive and vibratory feedback potentials in Brain-Computer Interfaces , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[75]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[76]  C. Neuper,et al.  Detection of Error Potentials During a Car-Game with Combined Continuous and Discrete Feedback , 2011 .

[77]  M. K. Goel,et al.  Cortical current density vs. surface EEG for event-related potential-based Brain-Computer Interface , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[78]  Ricardo Chavarriaga,et al.  Minimizing calibration time of single-trial recognition of error potentials in brain-computer interfaces , 2011 .

[79]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[80]  George K. Matsopoulos,et al.  Classification of Error-Related Negativity (ERN) and Positivity (Pe) potentials using kNN and Support Vector Machines , 2011, Comput. Biol. Medicine.

[81]  Nico M Schmidt,et al.  Online detection of error-related potentials boosts the performance of mental typewriters , 2012, BMC Neuroscience.

[82]  Ricardo Chavarriaga,et al.  Learning dictionaries of spatial and temporal EEG primitives for brain-computer interfaces , 2011, ICML 2011.

[83]  Xavier Artusi,et al.  Performance of a Simulated Adaptive BCI Based on Experimental Classification of Movement-Related and Error Potentials , 2011, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[84]  Ricardo Chavarriaga,et al.  Latency correction of error potentials between different experiments reduces calibration time for single-trial classification , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[85]  Wolfgang Rosenstiel,et al.  Online use of error-related potentials in healthy users and people with severe motor impairment increases performance of a P300-BCI , 2012, Clinical Neurophysiology.

[86]  Bertrand Olivier,et al.  Objective and subjective evaluation of online error correction during P300-based spelling , 2012 .

[87]  Emmanuel Maby,et al.  Objective and Subjective Evaluation of Online Error Correction during P300-Based Spelling , 2012, Adv. Hum. Comput. Interact..

[88]  John J. B. Allen,et al.  Theta lingua franca: a common mid-frontal substrate for action monitoring processes. , 2012, Psychophysiology.

[89]  Matthew B. Pontifex,et al.  Alterations in error-related brain activity and post-error behavior over time , 2012, Brain and Cognition.

[90]  Roland Siegwart,et al.  Anticipation- and error-related EEG signals during realistic human-machine interaction: A study on visual and tactile feedback , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[91]  T O Zander,et al.  Context-aware brain–computer interfaces: exploring the information space of user, technical system and environment , 2012, Journal of neural engineering.

[92]  R. Sandra,et al.  Designing spatial filters based on neuroscience theories to improve error-related potential classification , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[93]  Jan R. Wessel,et al.  Error awareness and the error-related negativity: evaluating the first decade of evidence , 2012, Front. Hum. Neurosci..

[94]  D. Mattia,et al.  Evaluation of the performances of different P300 based brain–computer interfaces by means of the efficiency metric , 2012, Journal of Neuroscience Methods.

[95]  Lucian Gheorghe,et al.  Improved recognition of error related potentials through the use of brain connectivity features , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[96]  Arne Robben,et al.  Towards the detection of error-related potentials and its integration in the context of a P300 speller brain-computer interface , 2012, Neurocomputing.

[97]  Sven Hoffmann,et al.  Predictive information processing in the brain: errors and response monitoring. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[98]  Vicenç Gómez,et al.  Adaptive Classification on Brain-Computer Interfaces Using Reinforcement Signals , 2012, Neural Computation.

[99]  Dominic Heger,et al.  Reliable subject-adapted recognition of EEG error potentials using limited calibration data , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[100]  I Iturrate,et al.  Task-dependent signal variations in EEG error-related potentials for brain–computer interfaces , 2013, Journal of neural engineering.

[101]  Maureen Clerc,et al.  An analysis of performance evaluation for motor-imagery based BCI , 2013, Journal of neural engineering.

[102]  Anna Weinberg,et al.  Biological Psychology , 2022 .

[103]  Elsa Andrea Kirchner,et al.  Classifier Transferability in the Detection of Error Related Potentials from Observation to Interaction , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[104]  R. Knight,et al.  Error-Monitoring and Post-Error Compensations: Dissociation between Perceptual Failures and Motor Errors with and without Awareness , 2013, The Journal of Neuroscience.

[105]  José del R. Millán,et al.  Inverse Solutions for Brain Computer Interface , 2013 .

[106]  Iñaki Iturrate,et al.  Shared control of a robot using EEG-based feedback signals , 2013, MLIS '13.

[107]  Iñaki Iturrate,et al.  Using frequency-domain features for the generalization of EEG error-related potentials among different tasks , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[108]  C. Mehring,et al.  Detection of Error Related Neuronal Responses Recorded by Electrocorticography in Humans during Continuous Movements , 2013, PloS one.

[109]  Tobias Kaufmann,et al.  Comparison of tactile, auditory, and visual modality for brain-computer interface use: a case study with a patient in the locked-in state , 2013, Front. Neurosci..

[110]  Lucian Gheorghe,et al.  Inferring driver's turning direction through detection of error related brain activity , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[111]  R Chavarriaga,et al.  Latency correction of event-related potentials between different experimental protocols. , 2014, Journal of neural engineering.

[112]  Tobias U. Hauser,et al.  The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization , 2014, NeuroImage.

[113]  Adrian G. Fischer,et al.  Neural mechanisms and temporal dynamics of performance monitoring , 2014, Trends in Cognitive Sciences.

[114]  Luca Mainardi,et al.  Performance measurement for brain–computer or brain–machine interfaces: a tutorial , 2014, Journal of neural engineering.

[115]  Pierre-Yves Oudeyer,et al.  Calibration-Free BCI Based Control , 2014, AAAI.

[116]  Tom Chau,et al.  A case study of linear classifiers adapted using imperfect labels derived from human event-related potentials , 2014, Pattern Recognit. Lett..