Development of a Guidance System for Motor Imagery Enhancement Using the Virtual Hand Illusion

Motor imagery (MI) is widely used to produce input signals for brain–computer interfaces (BCI) due to the similarities between MI-BCI and the planning–execution cycle. Despite its usefulness, MI tasks can be ambiguous to users and MI produces weaker cortical signals than motor execution. Existing MI guidance systems, which have been reported to provide visual guidance for MI and enhance MI, still have limitations: insufficient immersion for MI or poor expandability to MI for another body parts. We propose a guidance system for MI enhancement that can immerse users in MI and will be easy to extend to other body parts and target motions with few physical constraints. To make easily extendable MI guidance system, the virtual hand illusion is applied to the MI guidance system with a motion tracking sensor. MI enhancement was evaluated in 11 healthy people by comparison with another guidance system and conventional motor commands for BCI. The results showed that the proposed MI guidance system produced an amplified cortical signal compared to pure MI (p < 0.017), and a similar cortical signal as those produced by both actual execution (p > 0.534) and an MI guidance system with the rubber hand illusion (p > 0.722) in the contralateral region. Therefore, we believe that the proposed MI guidance system with the virtual hand illusion is a viable alternative to existing MI guidance systems in various applications with MI-BCI.

[1]  Huosheng Hu,et al.  A Self-Paced Motor Imagery Based Brain-Computer Interface for Robotic Wheelchair Control , 2011, Clinical EEG and neuroscience.

[2]  M. Hallett,et al.  Classifying EEG signals preceding right hand, left hand, tongue, and right foot movements and motor imageries , 2008, Clinical Neurophysiology.

[3]  Jinchang Ren,et al.  EEG-Based Brain-Computer Interfaces Using Motor-Imagery: Techniques and Challenges , 2019, Sensors.

[4]  Federico Schena,et al.  The Moving Rubber Hand Illusion Reveals that Explicit Sense of Agency for Tapping Movements Is Preserved in Functional Movement Disorders , 2017, Front. Hum. Neurosci..

[5]  Markus Neuhäuser,et al.  Wilcoxon Signed Rank Test , 2006 .

[6]  M. Reiner,et al.  Perspectives and possible applications of the rubber hand and virtual hand illusion in non-invasive rehabilitation: Technological improvements and their consequences , 2014, Neuroscience & Biobehavioral Reviews.

[7]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[8]  Sungho Jo,et al.  Improving performance in motor imagery BCI-based control applications via virtually embodied feedback , 2020, Comput. Biol. Medicine.

[9]  S. Swinnen,et al.  Neural correlates of action: Comparing meta-analyses of imagery, observation, and execution , 2018, Neuroscience & Biobehavioral Reviews.

[10]  G. Pfurtscheller,et al.  EEG-based discrimination between imagination of right and left hand movement. , 1997, Electroencephalography and clinical neurophysiology.

[11]  M Congedo,et al.  A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update , 2018, Journal of neural engineering.

[12]  Jonghyun Kim,et al.  A Novel Movement Intention Detection Method for Neurorehabilitation Brain-Computer Interface System , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[13]  G. Pfurtscheller,et al.  Functional brain imaging based on ERD/ERS , 2001, Vision Research.

[14]  Yijun Wang,et al.  Practical Designs of Brain–Computer Interfaces Based on the Modulation of EEG Rhythms , 2009 .

[15]  Maria V. Sanchez-Vives,et al.  Towards a Digital Body: The Virtual Arm Illusion , 2008, Frontiers in human neuroscience.

[16]  Jonathan D. Nelson,et al.  Human cortical representations for reaching: Mirror neurons for execution, observation, and imagery , 2007, NeuroImage.

[17]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[18]  T. Mulder.  Motor imagery and action observation: cognitive tools for rehabilitation , 2007, Journal of Neural Transmission.

[19]  Andreas Kalckert,et al.  The Onset Time of the Ownership Sensation in the Moving Rubber Hand Illusion , 2017, Front. Psychol..

[20]  F. Liarokapis,et al.  Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment , 2019, Front. Hum. Neurosci..

[21]  C. Nam,et al.  Event-related (De)synchronization (ERD/ERS) during motor imagery tasks: Implications for brain–computer interfaces , 2011 .

[22]  Seunghue Oh,et al.  Toward Comparison of Cortical Activation with Different Motor Learning Methods Using Event-Related Design: EEG-fNIRS Study , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[23]  I. Toni,et al.  Distinct roles for alpha- and beta-band oscillations during mental simulation of goal-directed actions. , 2014, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[25]  Eran Dayan,et al.  Alpha and Beta Band Event-Related Desynchronization Reflects Kinematic Regularities , 2015, The Journal of Neuroscience.

[26]  Minsu Song,et al.  A Paradigm to Enhance Motor Imagery Using Rubber Hand Illusion Induced by Visuo-Tactile Stimulus , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  C. Winstein,et al.  The Mirror Neuron System: A Neural Substrate for Methods in Stroke Rehabilitation , 2010, Neurorehabilitation and neural repair.

[28]  Gail F. Fahoome,et al.  Friedman's Test† , 2014 .

[29]  Glyn Humphreys,et al.  Mu rhythm desynchronization reveals motoric influences of hand action on object recognition , 2013, Front. Hum. Neurosci..

[30]  J. Wolpaw,et al.  Mu and Beta Rhythm Topographies During Motor Imagery and Actual Movements , 2004, Brain Topography.

[31]  Kup-Sze Choi,et al.  Improving the discrimination of hand motor imagery via virtual reality based visual guidance , 2016, Comput. Methods Programs Biomed..

[32]  Maria V. Sanchez-Vives,et al.  Virtual Hand Illusion Induced by Visuomotor Correlations , 2010, PloS one.

[33]  Jin-Chern Chiou,et al.  A Comparison of Independent Event-Related Desynchronization Responses in Motor-Related Brain Areas to Movement Execution, Movement Imagery, and Movement Observation , 2016, PloS one.

[34]  Sarah E Williams.  Comparing movement imagery and action observation as techniques to increase imagery ability , 2019, Psychology of Sport and Exercise.

[35]  Gonzalo G. de Polavieja,et al.  Active control as evidence in favor of sense of ownership in the moving Virtual Hand Illusion , 2019, Consciousness and Cognition.

[36]  Mirta Fiorio,et al.  Judging the position of the artificial hand induces a “visual” drift towards the real one during the rubber hand illusion , 2018, Scientific Reports.

[37]  Michael Yu Wang,et al.  An EEG/EMG/EOG-Based Multimodal Human-Machine Interface to Real-Time Control of a Soft Robot Hand , 2019, Front. Neurorobot..

[38]  J. Wolpaw,et al.  Brain–computer interfaces in neurological rehabilitation , 2008, The Lancet Neurology.

[39]  Mel Slater,et al.  Inducing a virtual hand ownership illusion through a brain–computer interface , 2009, Neuroreport.

[40]  K. Nakazawa,et al.  Difference in phase modulation of corticospinal excitability during the observation of the action of walking, with and without motor imagery , 2017, Neuroreport.

[41]  Brendan Z. Allison,et al.  Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction , 2013 .

[42]  Sungho Jo,et al.  Observing Actions Through Immersive Virtual Reality Enhances Motor Imagery Training , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[43]  M. Hallett,et al.  Prediction of human voluntary movement before it occurs , 2011, Clinical Neurophysiology.

[44]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[45]  Milan Simic,et al.  Mu-Beta event-related (de)synchronization and EEG classification of left-right foot dorsiflexion kinaesthetic motor imagery for BCI , 2020, PloS one.

[46]  G. Pfurtscheller,et al.  Motor imagery activates primary sensorimotor area in humans , 1997, Neuroscience Letters.

[47]  R N Vigário,et al.  Extraction of ocular artefacts from EEG using independent component analysis. , 1997, Electroencephalography and clinical neurophysiology.

[48]  R P Lesser,et al.  Functional significance of the mu rhythm of human cortex: an electrophysiologic study with subdural electrodes. , 1993, Electroencephalography and clinical neurophysiology.

[49]  Chao Li,et al.  A Brain-Machine Interface Based on ERD/ERS for an Upper-Limb Exoskeleton Control , 2016, Sensors.