Human-agent co-adaptation using error-related potentials

OBJECTIVE Error-related potentials (ErrP) have been proposed as an intuitive feedback signal decoded from the ongoing electroencephalogram (EEG) of a human observer for improving human-robot interaction (HRI). While recent demonstrations of this approach have successfully studied the use of ErrPs as a teaching signal for robot skill learning, so far, no efforts have been made towards HRI scenarios where mutual adaptations between human and robot are expected or required. These are collaborative or social interactive scenarios without predefined dominancy of the human partner and robots being perceived as intentional agents. Here we explore the usability of ErrPs as a feedback signal from the human for mediating co-adaptation in human-robot interaction. APPROACH We experimentally demonstrate ErrPs-based mediation of co-adaptation in a human-robot interaction study where successful interaction depended on co-adaptive convergence to a consensus between them. While subjects adapted to the robot by reflecting upon its behavior, the robot adapted its behavior based on ErrPs decoded online from the human partner's ongoing EEG. MAIN RESULTS ErrPs were decoded online in single trial with an avg. accuracy of 81.8%  ±  8.0% across 13 subjects, which was sufficient for effective adaptation of robot behavior. Successful co-adaptation was demonstrated by significant improvements in human-robot interaction efficacy and efficiency, and by the robot behavior that emerged during co-adaptation. These results indicate the potential of ErrPs as a useful feedback signal for mediating co-adaptation in human-robot interaction as demonstrated in a practical example. SIGNIFICANCE As robots become more widely embedded in society, methods for aligning them to human expectations and conventions will become increasingly important in the future. In this quest, ErrPs may constitute a promising complementary feedback signal for guiding adaptations towards human preferences. In this paper we extended previous research to less constrained HRI scenarios where mutual adaptations between human and robot are expected or required.

[1]  Martin Spüler,et al.  Error-related potentials during continuous feedback: using EEG to detect errors of different type and severity , 2015, Front. Hum. Neurosci..

[2]  Guillaume Gibert,et al.  OpenViBE: An Open-Source Software Platform to Design, Test, and Use BrainComputer Interfaces in Real and Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[3]  Joshua W. Brown,et al.  Medial prefrontal cortex as an action-outcome predictor , 2011, Nature Neuroscience.

[4]  Luis Montesano,et al.  Single trial recognition of error-related potentials during observation of robot operation , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[5]  Wolfgang Rosenstiel,et al.  Online Adaptation of a c-VEP Brain-Computer Interface(BCI) Based on Error-Related Potentials and Unsupervised Learning , 2012, PloS one.

[6]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[7]  Uri Hadar,et al.  The timing of shifts of head postures during conservation , 1984 .

[8]  Ricardo Chavarriaga,et al.  Errare machinale est: the use of error-related potentials in brain-machine interfaces , 2014, Front. Neurosci..

[9]  José del R. Millán,et al.  Error-Related EEG Potentials Generated During Simulated Brain–Computer Interaction , 2008, IEEE Transactions on Biomedical Engineering.

[10]  Jason Omedes,et al.  Factors that affect error potentials during a grasping task: toward a hybrid natural movement decoding BCI , 2018, Journal of neural engineering.

[11]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[12]  L. Montesano,et al.  Analysis and asynchronous detection of gradually unfolding errors during monitoring tasks , 2015, Journal of neural engineering.

[13]  Frank Kirchner,et al.  Intrinsic interactive reinforcement learning – Using error-related potentials for real world human-robot interaction , 2017, Scientific Reports.

[14]  M. Ullsperger,et al.  Neurophysiology of performance monitoring and adaptive behavior. , 2014, Physiological reviews.

[15]  Stefan Haufe,et al.  Single-trial analysis and classification of ERP components — A tutorial , 2011, NeuroImage.

[16]  Cuntai Guan,et al.  Learning from other subjects helps reducing Brain-Computer Interface calibration time , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  I Iturrate,et al.  Task-dependent signal variations in EEG error-related potentials for brain–computer interfaces , 2013, Journal of neural engineering.

[18]  Gordon Cheng,et al.  A neuro-based method for detecting context-dependent erroneous robot action , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[19]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[20]  Joseph DelPreto,et al.  Correcting robot mistakes in real time using EEG signals , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[21]  E. Procyk,et al.  Expectations, gains, and losses in the anterior cingulate cortex , 2007, Cognitive, affective & behavioral neuroscience.

[22]  David Goodman,et al.  Performance Monitoring in the Anterior Cingulate is Not All Error Related: Expectancy Deviation and the Representation of Action-Outcome Associations , 2007, Journal of Cognitive Neuroscience.

[23]  Jorge Dias,et al.  Designing an artificial attention system for social robots , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[25]  Karl J. Friston,et al.  The mismatch negativity: A review of underlying mechanisms , 2009, Clinical Neurophysiology.

[26]  Wolfgang Rosenstiel,et al.  Online use of error-related potentials in healthy users and people with severe motor impairment increases performance of a P300-BCI , 2012, Clinical Neurophysiology.

[27]  Elsa Andrea Kirchner,et al.  Handling Few Training Data: Classifier Transfer Between Different Types of Error-Related Potentials , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[28]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[29]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[30]  K.-R. Muller,et al.  Boosting bit rates and error detection for the classification of fast-paced motor commands based on single-trial EEG analysis , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  J. Hohnsbein,et al.  ERP components on reaction errors and their functional significance: a tutorial , 2000, Biological Psychology.

[32]  Nico M Schmidt,et al.  Online detection of error-related potentials boosts the performance of mental typewriters , 2012, BMC Neuroscience.

[33]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[34]  H. Bekkering,et al.  Modulation of activity in medial frontal and motor cortices during error observation , 2004, Nature Neuroscience.

[35]  Ricardo Chavarriaga,et al.  Minimizing calibration time using inter-subject information of single-trial recognition of error potentials in brain-computer interfaces , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[36]  R Chavarriaga,et al.  Learning From EEG Error-Related Potentials in Noninvasive Brain-Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  Ricardo Chavarriaga,et al.  Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control , 2015, Scientific Reports.

[38]  R. Homan,et al.  Cerebral location of international 10-20 system electrode placement. , 1987, Electroencephalography and clinical neurophysiology.

[39]  G. Pfurtscheller,et al.  EEG-based communication: presence of an error potential , 2000, Clinical Neurophysiology.

[40]  Christa Neuper,et al.  Error potential detection during continuous movement of an artificial arm controlled by brain–computer interface , 2012, Medical & Biological Engineering & Computing.

[41]  G. Pfurtscheller,et al.  A fully automated correction method of EOG artifacts in EEG recordings , 2007, Clinical Neurophysiology.

[42]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[43]  Gordon Cheng,et al.  A Simple and Practical Sensorimotor EEG Device for Recording in Patients with Special Needs , 2017, NEUROTECHNIX.

[44]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[45]  Guillaume Gibert,et al.  xDAWN Algorithm to Enhance Evoked Potentials: Application to Brain–Computer Interface , 2009, IEEE Transactions on Biomedical Engineering.