Coordinating With a Robot Partner Affects Neural Processing Related to Action Monitoring

Robots start to play a role in our social landscape, and they are progressively becoming responsive, both physically and socially. It begs the question of how humans react to and interact with robots in a coordinated manner and what the neural underpinnings of such behavior are. This exploratory study aims to understand the differences in human-human and human-robot interactions at a behavioral level and from a neurophysiological perspective. For this purpose, we adapted a collaborative dynamical paradigm from the literature. We asked 12 participants to hold two corners of a tablet while collaboratively guiding a ball around a circular track either with another participant or a robot. In irregular intervals, the ball was perturbed outward creating an artificial error in the behavior, which required corrective measures to return to the circular track again. Concurrently, we recorded electroencephalography (EEG). In the behavioral data, we found an increased velocity and positional error of the ball from the track in the human-human condition vs. human-robot condition. For the EEG data, we computed event-related potentials. We found a significant difference between human and robot partners driven by significant clusters at fronto-central electrodes. The amplitudes were stronger with a robot partner, suggesting a different neural processing. All in all, our exploratory study suggests that coordinating with robots affects action monitoring related processing. In the investigated paradigm, human participants treat errors during human-robot interaction differently from those made during interactions with other humans. These results can improve communication between humans and robot with the use of neural activity in real-time.

[1]  Agnieszka Wykowska,et al.  Robot Gaze Behavior Affects Honesty in Human-Robot Interaction , 2021, Frontiers in Artificial Intelligence.

[2]  B. Wiederhold The Ascent of Social Robots , 2021, Cyberpsychology Behav. Soc. Netw..

[3]  Benedikt V. Ehinger,et al.  Regression-based analysis of combined EEG and eye-tracking data: Theory and applications , 2021, Journal of vision.

[4]  Jan R. Wessel,et al.  #EEGManyLabs: Investigating the replicability of influential EEG experiments , 2020, Cortex.

[5]  Leonard S. Peperkoorn,et al.  The prevalence of dyads in social life , 2020, PloS one.

[6]  Gordon Cheng,et al.  Neuroengineering challenges of fusing robotics and neuroscience , 2020, Science Robotics.

[7]  F. Ciardo,et al.  ERP markers of action planning and outcome monitoring in human - robot interaction. , 2020, Acta psychologica.

[8]  A. Kingstone,et al.  Humans share task load with a computer partner if (they believe that) it acts human-like. , 2020, Acta psychologica.

[9]  Daniel Polani,et al.  Tilting Together: An Information-Theoretic Characterization of Behavioral Roles in Rhythmic Dyadic Interaction , 2020, Frontiers in Human Neuroscience.

[10]  Peter König,et al.  Hyperscanning: A Valid Method to Study Neural Inter-brain Underpinnings of Social Interaction , 2020, Frontiers in Human Neuroscience.

[11]  Olaf Dimigen,et al.  Optimizing the ICA-based removal of ocular EEG artifacts from free viewing experiments , 2020, NeuroImage.

[12]  Samuel A. Nastase,et al.  Keep it real: rethinking the primacy of experimental control in cognitive neuroscience , 2020, NeuroImage.

[13]  Gordon Cheng,et al.  A computational model of human decision making and learning for assessment of co-adaptation in neuro-adaptive human-robot interaction , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).

[14]  Martin Wetzels,et al.  Trust in humanoid robots: implications for services marketing , 2019, Journal of Services Marketing.

[15]  Arianna Curioni,et al.  Reciprocal information flow and role distribution support joint action coordination , 2019, Cognition.

[16]  E. Redcay,et al.  Using second-person neuroscience to elucidate the mechanisms of social interaction , 2019, Nature Reviews Neuroscience.

[17]  Ricardo Chavarriaga,et al.  Inferring subjective preferences on robot trajectories using EEG signals , 2019, 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER).

[18]  Catherine Perrodin,et al.  Are We Ready for Real-world Neuroscience? , 2019, Journal of Cognitive Neuroscience.

[19]  Kenneth Kreutz-Delgado,et al.  ICLabel: An automated electroencephalographic independent component classifier, dataset, and website , 2019, NeuroImage.

[20]  Gordon Cheng,et al.  A Feasibility Study for Validating Robot Actions Using EEG-Based Error-Related Potentials , 2018, International Journal of Social Robotics.

[21]  Takako Fujioka,et al.  Performance monitoring of self and other in a turn-taking piano duet: A dual-EEG study , 2018, Social neuroscience.

[22]  Benedikt V. Ehinger,et al.  The Social Situation Affects How We Process Feedback About Our Actions , 2018, bioRxiv.

[23]  Gordon Cheng,et al.  Human-agent co-adaptation using error-related potentials , 2018, Journal of neural engineering.

[24]  N. Sebanz,et al.  Joint action coordination in expert-novice pairs: Can experts predict novices’ suboptimal timing? , 2018, Cognition.

[25]  A. Diamond Robots and Computers Enhance Us More Than They Replace Us , 2018, The American Economist.

[26]  Lucian Gheorghe,et al.  Decoding Neural Correlates of Cognitive States to Enhance Driving Experience , 2018, IEEE Transactions on Emerging Topics in Computational Intelligence.

[27]  Olaf Dimigen,et al.  Unfold: an integrated toolbox for overlap correction, non-linear modeling, and regression-based EEG analysis , 2018, bioRxiv.

[28]  H. Blume,et al.  Effect- and Performance-Based Auditory Feedback on Interpersonal Coordination , 2018, Front. Psychol..

[29]  Vijay Kumar,et al.  The grand challenges of Science Robotics , 2018, Science Robotics.

[30]  Mordechai Ben-Ari,et al.  Robots and Their Applications , 2018 .

[31]  Katia Sycara,et al.  The role of trust in human-robot interaction , 2018 .

[32]  Frank Kirchner,et al.  Intrinsic interactive reinforcement learning – Using error-related potentials for real world human-robot interaction , 2017, Scientific Reports.

[33]  Cordula Vesper,et al.  Modulating Action Duration to Establish Nonconventional Communication , 2017, Journal of experimental psychology. General.

[34]  Joseph DelPreto,et al.  Correcting robot mistakes in real time using EEG signals , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Sylvain Baillet,et al.  Magnetoencephalography for brain electrophysiology and imaging , 2017, Nature Neuroscience.

[36]  M. A. MacIver,et al.  Neuroscience Needs Behavior: Correcting a Reductionist Bias , 2017, Neuron.

[37]  E. Broadbent Interactions With Robots: The Truths We Reveal About Ourselves , 2017, Annual review of psychology.

[38]  Günther Knoblich,et al.  Joint Action in Humans: A Model for Human-Robot Interactions , 2017 .

[39]  G. Cheng,et al.  Embodied artificial agents for understanding human social cognition , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[40]  Thomas B. Sheridan,et al.  Human–Robot Interaction , 2016, Hum. Factors.

[41]  O. Khatib,et al.  The rise of social robots : a review of the recent literature , 2016 .

[42]  R Chavarriaga,et al.  EEG-based decoding of error-related brain activity in a real-world driving task , 2015, Journal of neural engineering.

[43]  Ricardo Chavarriaga,et al.  Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control , 2015, Scientific Reports.

[44]  Erich Schröger,et al.  Digital filter design for electrophysiological data – a practical approach , 2015, Journal of Neuroscience Methods.

[45]  Peter König,et al.  Predictions of Visual Content across Eye Movements and Their Modulation by Inferred Information , 2015, The Journal of Neuroscience.

[46]  Daniel C. Richardson,et al.  Progress in Joint-Action Research , 2021, CogSci.

[47]  G. Pezzulo,et al.  Human Sensorimotor Communication: A Theory of Signaling in Online Social Interactions , 2013, PloS one.

[48]  K. Vogeley,et al.  Toward a second-person neuroscience 1 , 2013, Behavioral and Brain Sciences.

[49]  Janeen D. Loehr,et al.  Monitoring Individual and Joint Action Outcomes in Duet Music Performance , 2013, Journal of Cognitive Neuroscience.

[50]  Holly A. Yanco,et al.  Impact of robot failures and feedback on real-time trust , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[51]  Ramin Khatami,et al.  Advanced EEG analysis using threshold-free cluster-enhancement and non-parametric statistics , 2013, NeuroImage.

[52]  Yoshio Matsumoto,et al.  How Do People Perceive and Trust a Lifelike Robot , 2013 .

[53]  Marco Ferrari,et al.  A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application , 2012, NeuroImage.

[54]  Carsten Zoll,et al.  The Social Role of Robots in the Future—Explorative Measurement of Hopes and Fears , 2011, Int. J. Soc. Robotics.

[55]  Cordula Vesper,et al.  A minimal architecture for joint action , 2010, Neural Networks.

[56]  John J. B. Allen,et al.  Prelude to and Resolution of an Error: EEG Phase Synchrony Reveals Cognitive Control Dynamics during Action Monitoring , 2009, The Journal of Neuroscience.

[57]  Bhaskar D. Rao,et al.  Newton method for the ICA mixture model , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[58]  Clay B. Holroyd,et al.  The feedback-related negativity reflects the binary evaluation of good versus bad outcomes , 2006, Biological Psychology.

[59]  H. Bekkering,et al.  Joint action: bodies and minds moving together , 2006, Trends in Cognitive Sciences.

[60]  Jonathan D. Cohen,et al.  The neural basis of error detection: conflict monitoring and the error-related negativity. , 2004, Psychological review.

[61]  H. Bekkering,et al.  Modulation of activity in medial frontal and motor cortices during error observation , 2004, Nature Neuroscience.

[62]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[63]  Matthew D. Lieberman,et al.  Does Rejection Hurt? An fMRI Study of Social Exclusion , 2003, Science.

[64]  W. Prinz,et al.  Representing others' actions: just like one's own? , 2003, Cognition.

[65]  C. Braun,et al.  Event-Related Brain Potentials Following Incorrect Feedback in a Time-Estimation Task: Evidence for a Generic Neural System for Error Detection , 1997, Journal of Cognitive Neuroscience.

[66]  S J Luck,et al.  Spatial filtering during visual search: evidence from human electrophysiology. , 1994, Journal of experimental psychology. Human perception and performance.

[67]  C. E. Rogers,et al.  Symbolic Description of Factorial Models for Analysis of Variance , 1973 .

[68]  Keh-Shew Lu,et al.  DIGITAL FILTER DESIGN , 1973 .