Embodied VR environment facilitates motor imagery brain-computer interface training

Motor imagery (MI) is the predominant control paradigm for brain–computer interfaces (BCIs). After sufficient effort is invested to the training, the accuracy of commands mediated by mental imagery of bodily movements grows to a satisfactory level. However, many issues with the MI-BCIs persist; e.g., low bit transfer rate, BCI illiteracy, sub-optimal training procedure. Especially the training process for the MI-BCIs requires improvements. Currently, the training has an inappropriate form, resulting in a high mental and temporal demand on the users (weeks of training are required for the control). This study aims at addressing the issues with the MI-BCI training. To support the learning process, an embodied training environment was created. Participants were placed into a virtual reality environment observed from a first-person view of a human-like avatar, and their rehearsal of MI actions was reflected by the corresponding movements performed by the avatar. Leveraging extension of the sense of ownership, agency, and self-location towards a non-body object (principles known from the rubber hand illusion and the body transfer illusions) has already been proven to help in producing stronger EEG correlates of MI. These principles were used to facilitate the MI-BCI training process for the first time. Performance of 30 healthy participants after two sessions of training was measured using an on-line BCI scenario. The group trained using our embodied VR environment gained significantly higher average accuracy for BCI actions (58.3%) than the control group, trained with a standard MI-BCI training protocol (52.9%).

[1]  H. Ehrsson,et al.  The moving rubber hand illusion revisited: Comparing movements and visuotactile stimulation to induce illusory ownership , 2014, Consciousness and Cognition.

[2]  V. Ramachandran,et al.  Projecting sensations to external objects: evidence from skin conductance response , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[3]  Antonio Frisoli,et al.  The Modulation of Ownership and Agency in the Virtual Hand Illusion under Visuotactile and Visuomotor Sensory Feedback , 2014, PRESENCE: Teleoperators and Virtual Environments.

[4]  Mel Slater,et al.  Inducing a virtual hand ownership illusion through a brain–computer interface , 2009, Neuroreport.

[5]  H. Ehrsson,et al.  Moving a Rubber Hand that Feels Like Your Own: A Dissociation of Ownership and Agency , 2012, Front. Hum. Neurosci..

[6]  Ricardo Chavarriaga,et al.  Heading for new shores! Overcoming pitfalls in BCI design. , 2017, Brain computer interfaces.

[7]  B. Dobkin Brain–computer interface technology as a tool to augment plasticity and outcomes for neurological rehabilitation , 2007, The Journal of physiology.

[8]  Reinhold Scherer,et al.  Mind the Traps! Design Guidelines for Rigorous BCI Experiments , 2018 .

[9]  Vera Kaiser,et al.  Cortical effects of user training in a motor imagery based brain–computer interface measured by fNIRS and EEG , 2014, NeuroImage.

[10]  S. Swinnen,et al.  Kinesthetic, but not visual, motor imagery modulates corticomotor excitability , 2005, Experimental Brain Research.

[11]  Olaf Blanke,et al.  Visual Feedback Dominates the Sense of Agency for Brain-Machine Actions , 2015, PloS one.

[12]  Karim Jerbi,et al.  Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy , 2015, Journal of Neuroscience Methods.

[13]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[14]  F Lotte,et al.  Advances in user-training for mental-imagery-based BCI control: Psychological and cognitive factors and their neural correlates. , 2016, Progress in brain research.

[15]  J. Wolpaw,et al.  Brain-Computer Interfaces: Principles and Practice , 2012 .

[16]  Anatole Lécuyer,et al.  Combining BCI with Virtual Reality: Towards New Applications and Improved BCI , 2012 .

[17]  Maria V. Sanchez-Vives,et al.  Inducing Illusory Ownership of a Virtual Body , 2009, Front. Neurosci..

[18]  Christian Mühl,et al.  Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design , 2013, Front. Hum. Neurosci..

[19]  G. Pfurtscheller,et al.  Graz brain-computer interface II: towards communication between humans and computers based on online classification of three different EEG patterns , 1996, Medical and Biological Engineering and Computing.

[20]  Yvonne de Kort,et al.  Is This My Hand I See Before Me? The Rubber Hand Illusion in Reality, Virtual Reality, and Mixed Reality , 2006, PRESENCE: Teleoperators and Virtual Environments.

[21]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[22]  Fabien Lotte,et al.  Why standard brain-computer interface (BCI) training protocols should be changed: an experimental study , 2016, Journal of neural engineering.

[23]  C. Spence,et al.  Psychologically induced cooling of a specific body part caused by the illusory ownership of an artificial counterpart , 2008, Proceedings of the National Academy of Sciences.

[24]  Shuichi Nishio,et al.  Humanlike robot hands controlled by brain activity arouse illusion of ownership in operators , 2013, Scientific Reports.

[25]  Vera Kaiser,et al.  Thinking Penguin: Multimodal Brain–Computer Interface Control of a VR Game , 2013, IEEE Transactions on Computational Intelligence and AI in Games.

[26]  Francisco Velasco-Álvarez,et al.  Free Virtual Navigation Using Motor Imagery Through an Asynchronous BrainComputer Interface , 2010, PRESENCE: Teleoperators and Virtual Environments.

[27]  Shuichi Nishio,et al.  Removal of proprioception by BCI raises a stronger body ownership illusion in control of a humanlike robot , 2016, Scientific Reports.

[28]  A. Pope,et al.  Biocybernetic system evaluates indices of operator engagement in automated task , 1995, Biological Psychology.

[29]  G. Pfurtscheller,et al.  Enhancement of left-right sensorimotor EEG differences during feedback-regulated motor imagery. , 1999, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[30]  Ferran Argelaguet,et al.  “Do You Feel in Control?”: Towards Novel Approaches to Characterise, Manipulate and Measure the Sense of Agency in Virtual Environments , 2018, IEEE Transactions on Visualization and Computer Graphics.

[31]  Fotis Liarokapis,et al.  Examining the effect of body ownership in immersive virtual and augmented reality environments , 2016, The Visual Computer.

[32]  G. Pfurtscheller,et al.  Motor imagery and action observation: Modulation of sensorimotor brain rhythms during mental control of a brain–computer interface , 2009, Clinical Neurophysiology.

[33]  Sergi Bermúdez i Badia,et al.  NeuRow: An Immersive VR Environment for Motor-Imagery Training with the Use of Brain-Computer Interfaces and Vibrotactile Feedback , 2016, PhyCS.

[34]  Fabien Lotte,et al.  Signal Processing Approaches to Minimize or Suppress Calibration Time in Oscillatory Activity-Based Brain–Computer Interfaces , 2015, Proceedings of the IEEE.

[35]  J. Hohwy,et al.  Explaining Away the Body: Experiences of Supernaturally Caused Touch and Touch on Non-Hand Objects within the Rubber Hand Illusion , 2010, PloS one.

[36]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[37]  Marieke Rohde,et al.  The Human Touch: Skin Temperature during the Rubber Hand Illusion in Manual and Automated Stroking Procedures , 2013, PloS one.

[38]  Eduardo Miranda,et al.  Guide to Brain-Computer Music Interfacing , 2014, Springer London.

[39]  A. Gevins,et al.  Mapping cognitive brain function with modern high-resolution electroencephalography , 1995, Trends in Neurosciences.

[40]  Maria V. Sanchez-Vives,et al.  Virtual Hand Illusion Induced by Visuomotor Correlations , 2010, PloS one.

[41]  M. Tsakiris Looking for Myself: Current Multisensory Input Alters Self-Face Recognition , 2008, PloS one.

[42]  Fotis Liarokapis,et al.  Investigating the effect of user profile during training for BCI-based games , 2017, 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games).

[43]  Maarten De Vos,et al.  Real-time EEG feedback during simultaneous EEG–fMRI identifies the cortical signature of motor imagery , 2015, NeuroImage.

[44]  G. Pfurtscheller,et al.  Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. , 2005, Brain research. Cognitive brain research.

[45]  Guillaume Gibert,et al.  OpenViBE: An Open-Source Software Platform to Design, Test, and Use BrainComputer Interfaces in Real and Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[46]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[47]  Tele Tan,et al.  3D visualization of movements can amplify motor cortex activation during subsequent motor imagery , 2015, Front. Hum. Neurosci..

[48]  Anatole Lécuyer,et al.  Passive Brain–Computer Interfaces , 2014 .

[49]  Mel Slater,et al.  The Sense of Embodiment in Virtual Reality , 2012, PRESENCE: Teleoperators and Virtual Environments.

[50]  Athanasios V. Vasilakos,et al.  Brain computer interface: control signals review , 2017, Neurocomputing.

[51]  Stefan Van der Stigchel,et al.  No consistent cooling of the real hand in the rubber hand illusion. , 2017, Acta psychologica.

[52]  S. Ainsworth DeFT: A Conceptual Framework for Considering Learning with Multiple Representations. , 2006 .

[53]  Shuichi Nishio,et al.  The Importance of Visual Feedback Design in BCIs; from Embodiment to Motor Imagery Learning , 2016, PloS one.

[54]  Toshiyuki Kondo,et al.  Effect of instructive visual stimuli on neurofeedback training for motor imagery-based brain-computer interface. , 2015, Human movement science.

[55]  Horst Bischof,et al.  Toward Self-Paced Brain–Computer Communication: Navigation Through Virtual Worlds , 2008, IEEE Transactions on Biomedical Engineering.

[56]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[57]  Carsten Mehring,et al.  Ownership and Agency of an Independent Supernumerary Hand Induced by an Imitation Brain-Computer Interface , 2015, PloS one.

[58]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[59]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[60]  Cuntai Guan,et al.  Regularizing Common Spatial Patterns to Improve BCI Designs: Unified Theory and New Algorithms , 2011, IEEE Transactions on Biomedical Engineering.

[61]  Jeremy D. Thorne,et al.  Embodied neurofeedback with an anthropomorphic robotic hand , 2016, Scientific Reports.

[62]  M. Slater,et al.  Measuring the Effects through Time of the Influence of Visuomotor and Visuotactile Synchronous Stimulation on a Virtual Body Ownership Illusion , 2014, Perception.

[63]  Nathan Evans,et al.  Shared electrophysiology mechanisms of body ownership and motor imagery , 2013, NeuroImage.

[64]  G. Rizzolatti,et al.  Activation of human primary motor cortex during action observation: a neuromagnetic study. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[65]  K. Hiraki,et al.  Rubber Hand Illusion under Delayed Visual Feedback , 2009, PloS one.

[66]  Roland H. Grabner,et al.  Oscillatory EEG Correlates of Arithmetic Strategies: A Training Study , 2012, Front. Psychology.

[67]  M. Jeannerod The mechanism of self-recognition in humans , 2003, Behavioural Brain Research.