Embodied neurofeedback with an anthropomorphic robotic hand

Neurofeedback-guided motor imagery training (NF-MIT) has been suggested as a promising therapy for stroke-induced motor impairment. Whereas much NF-MIT research has aimed at signal processing optimization, the type of sensory feedback given to the participant has received less attention. Often the feedback signal is highly abstract and not inherently coupled to the mental act performed. In this study, we asked whether an embodied feedback signal is more efficient for neurofeedback operation than a non-embodiable feedback signal. Inspired by the rubber hand illusion, demonstrating that an artificial hand can be incorporated into one’s own body scheme, we used an anthropomorphic robotic hand to visually guide the participants’ motor imagery act and to deliver neurofeedback. Using two experimental manipulations, we investigated how a participant’s neurofeedback performance and subjective experience were influenced by the embodiability of the robotic hand, and by the neurofeedback signal’s validity. As pertains to embodiment, we found a promoting effect of robotic-hand embodiment in subjective, behavioral, electrophysiological and electrodermal measures. Regarding neurofeedback signal validity, we found some differences between real and sham neurofeedback in terms of subjective and electrodermal measures, but not in terms of behavioral and electrophysiological measures. This study motivates the further development of embodied feedback signals for NF-MIT.

[1]  Anton Nijholt,et al.  Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-World Applications , 2012 .

[2]  L. Moja,et al.  Constraint-Induced Movement Therapy for Upper Extremities in People With Stroke , 2015, The Cochrane database of systematic reviews.

[3]  Albert Jin Chung,et al.  Perception of Body Ownership Is Driven by Bayesian Sensory Inference , 2015, PloS one.

[4]  P. Venables,et al.  Publication recommendations for electrodermal measurements. , 1981 .

[5]  H. Ehrsson,et al.  The moving rubber hand illusion revisited: Comparing movements and visuotactile stimulation to induce illusory ownership , 2014, Consciousness and Cognition.

[6]  F. Vignemont Embodiment, ownership and disownership , 2011, Consciousness and Cognition.

[7]  Marcia K. Johnson,et al.  Phenomenal characteristics of memories for perceived and imagined autobiographical events. , 1988, Journal of experimental psychology. General.

[8]  A. Engel,et al.  Trial-by-Trial Coupling of Concurrent Electroencephalogram and Functional Magnetic Resonance Imaging Identifies the Dynamics of Performance Monitoring , 2005, The Journal of Neuroscience.

[9]  M. Molinari,et al.  Brain–computer interface boosts motor imagery practice during stroke recovery , 2015, Annals of neurology.

[10]  Akio Gofuku,et al.  Nonimmersive virtual reality mirror visual feedback therapy and its application for the treatment of complex regional pain syndrome: an open-label pilot study. , 2010, Pain medicine.

[11]  Jeremy D. Thorne,et al.  Interplay of Agency and Ownership: The Intentional Binding and Rubber Hand Illusion Paradigm Combined , 2014, PloS one.

[12]  Scott Makeig,et al.  BCILAB: a platform for brain–computer interface development , 2013, Journal of neural engineering.

[13]  O. Güntürkün,et al.  Laterality in the rubber hand illusion , 2011, Laterality.

[14]  C. Moore,et al.  Movement and the Rubber Hand Illusion , 2009, Perception.

[15]  Maarten De Vos,et al.  Lateralization patterns of covert but not overt movements change with age: An EEG neurofeedback study , 2015, NeuroImage.

[16]  A. Seth Interoceptive inference, emotion, and the embodied self , 2013, Trends in Cognitive Sciences.

[17]  J. Ushiba,et al.  Daily training with realistic visual feedback improves reproducibility of event-related desynchronisation following hand motor imagery , 2013, Clinical Neurophysiology.

[18]  Raoul M Bongers,et al.  Systematic review of the effectiveness of mirror therapy in upper extremity function , 2009, Disability and rehabilitation.

[19]  Brendan Z. Allison,et al.  Is It Significant? Guidelines for Reporting BCI Performance , 2012 .

[20]  Thomas Metzinger,et al.  Self models , 2007, Scholarpedia.

[21]  Guilherme Borges,et al.  Correlations between motor and sensory functions in upper limb chronic hemiparetics after stroke. , 2011, Arquivos de neuro-psiquiatria.

[22]  M. Benedek,et al.  Decomposition of skin conductance data by means of nonnegative deconvolution , 2010, Psychophysiology.

[23]  V. Ramachandran,et al.  The use of visual feedback, in particular mirror visual feedback, in restoring brain function. , 2009, Brain : a journal of neurology.

[24]  Christian Mühl,et al.  Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design , 2013, Front. Hum. Neurosci..

[25]  Derick T Wade,et al.  Somatosensory recovery: A longitudinal study of the first 6 months after unilateral stroke , 2007, Disability and rehabilitation.

[26]  M. Mirshoja,et al.  Comparison of Fine Motor Skills in Patients With Chronic Stroke in Final Stages of Bronestrum and Healthy Adults , 2015 .

[27]  Maria V. Sanchez-Vives,et al.  Inducing Illusory Ownership of a Virtual Body , 2009, Front. Neurosci..

[28]  Motor imagery in stroke patients, or plegic patients with spinal cord or peripheral diseases , 2012, Acta neurologica Scandinavica.

[29]  R. Passingham,et al.  That's My Hand! Activity in Premotor Cortex Reflects Feeling of Ownership of a Limb , 2004, Science.

[30]  Howard Rosenbaum,et al.  Effects of reading proficiency on embedded stem priming in primary school children , 2021 .

[31]  P. Langhorne,et al.  Motor recovery after stroke: a systematic review , 2009, The Lancet Neurology.

[32]  E. Taub,et al.  Constraint Induced Movement Techniques To Facilitate Upper Extremity Use in Stroke Patients. , 1997, Topics in stroke rehabilitation.

[33]  Bert Otten,et al.  Motor imagery ability in stroke patients: the relationship between implicit and explicit motor imagery measures , 2013, Front. Hum. Neurosci..

[34]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[35]  H. Ehrsson,et al.  Moving a Rubber Hand that Feels Like Your Own: A Dissociation of Ownership and Agency , 2012, Front. Hum. Neurosci..

[36]  Shuichi Nishio,et al.  Humanlike robot hands controlled by brain activity arouse illusion of ownership in operators , 2013, Scientific Reports.

[37]  T. Metzinger How does the brain encode epistemic reliability? Perceptual presence, phenomenal transparency, and counterfactual richness , 2014, Cognitive neuroscience.

[38]  Felix Blankenburg,et al.  Minimal self-models and the free energy principle , 2013, Front. Hum. Neurosci..

[39]  R. Firth Function , 1955, Yearbook of Anthropology.

[40]  L. Cohen,et al.  Brain–machine interfaces in neurorehabilitation of stroke , 2015, Neurobiology of Disease.

[41]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[42]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[43]  M. Benedek,et al.  A continuous measure of phasic electrodermal activity , 2010, Journal of Neuroscience Methods.

[44]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[45]  Maarten De Vos,et al.  Real-time EEG feedback during simultaneous EEG–fMRI identifies the cortical signature of motor imagery , 2015, NeuroImage.

[46]  S. Debener,et al.  Cross-Modal Phase Reset Predicts Auditory Task Performance in Humans , 2011, The Journal of Neuroscience.

[47]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[48]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[49]  V. Demarin,et al.  Motor and cognitive impairment after stroke. , 2011, Acta clinica Croatica.

[50]  G. Ben-Shakhar,et al.  Publication recommendations for electrodermal measurements. , 1981, Psychophysiology.

[51]  Axel Cleeremans,et al.  New frontiers in the rubber hand experiment: when a robotic hand becomes one’s own , 2014, Behavior Research Methods.

[52]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[53]  J. Liepert,et al.  Reduced Upper Limb Sensation Impairs Mental Chronometry for Motor Imagery After Stroke , 2012, Neurorehabilitation and neural repair.

[54]  Mel Slater,et al.  Inducing a virtual hand ownership illusion through a brain–computer interface , 2009, Neuroreport.

[55]  G. Savelsbergh,et al.  Reflections on Mirror Therapy , 2015, Neurorehabilitation and neural repair.

[56]  V. Caggiano,et al.  Proprioceptive Feedback and Brain Computer Interface (BCI) Based Neuroprostheses , 2012, PloS one.

[57]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[58]  L. Avanzino,et al.  Training based on mirror visual feedback influences transcallosal communication , 2014, The European journal of neuroscience.

[59]  Marie Johnston,et al.  Mental practice with motor imagery in stroke recovery: randomized controlled trial of efficacy , 2011, Brain : a journal of neurology.

[60]  Shuichi Nishio,et al.  Effect of biased feedback on motor imagery learning in BCI-teleoperation system , 2014, Front. Syst. Neurosci..

[61]  P. Haggard,et al.  The rubber hand illusion revisited: visuotactile integration and self-attribution. , 2005, Journal of experimental psychology. Human perception and performance.

[62]  J. Baron,et al.  Motor Imagery: A Backdoor to the Motor System After Stroke? , 2006, Stroke.

[63]  Bernhard Schölkopf,et al.  Closing the sensorimotor loop: Haptic feedback facilitates decoding of arm movement imagery , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[64]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .