Disruption by virtual reality of the cortical oscillations related to visuotactile integration during an embodiment process

Virtual reality (VR) enables fast, free, and highly controllable experimental body image setting. Illusions pertaining to a body, like the rubber hand illusion (RHI), can be easily conducted in VR settings, and some phenomena, such as full-body illusions, are only realized in virtual environments. However, the multisensory integration process in VR is not yet fully understood, and we must clarify the limitations and whether specific phenomena can also occur in real life or only in VR settings. One useful investigative approach is measuring brain activities during a psychological experiment. Electroencephalography (EEG) oscillatory activities provide insight into the human multisensory integration process. Unfortunately, the data can be vulnerable to VR noise, which causes measurement and analytical difficulties for EEG data recorded in VR environments. Here, we took care to provide an experimental RHI setting using a head-mounted display, which provided a VR visual space and VR dummy hand along with EEG measurements. We compared EEG data taken in both real and VR environments and observed the gamma and theta band oscillatory activities. Ultimately, we saw statistically significant differences between congruent (RHI) and incongruent (not RHI) conditions in the real environment, which agrees with previous studies. No difference in the VR condition could be observed, suggesting that the VR setting itself altered the perceptual and sensory integration mechanisms. Thus, we must model this difference between real and VR settings whenever we use VR to investigate our bodily self-perception.

[1]  Nathan Evans,et al.  Shared electrophysiology mechanisms of body ownership and motor imagery , 2013, NeuroImage.

[2]  Borís Burle,et al.  Rostral Cingulate Zone and correct response monitoring: ICA and source localization evidences for the unicity of correct- and error-negativities , 2010, NeuroImage.

[3]  T. Cummins,et al.  Theta oscillations are affected by amnestic mild cognitive impairment and cognitive load. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  E. Halgren,et al.  Intracranial Cortical Responses during Visual–Tactile Integration in Humans , 2014, The Journal of Neuroscience.

[5]  R. Nigbur,et al.  Theta power as a marker for cognitive interference , 2011, Clinical Neurophysiology.

[6]  Mel Slater,et al.  How to Build an Embodiment Lab: Achieving Body Representation Illusions in Virtual Reality , 2014, Front. Robot. AI.

[7]  Noriaki Kanayama,et al.  Crossmodal effect with rubber hand illusion and gamma-band activity. , 2007, Psychophysiology.

[8]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[9]  Giulio Jacucci,et al.  Manipulating Bodily Presence Affects Cross-Modal Spatial Attention: A Virtual-Reality-Based ERP Study , 2017, Front. Hum. Neurosci..

[10]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[11]  O. Bertrand,et al.  Oscillatory gamma activity in humans and its role in object representation , 1999, Trends in Cognitive Sciences.

[12]  J. Maunsell,et al.  Touching a Rubber Hand: Feeling of Body Ownership Is Associated with Activity in Multisensory Brain Areas , 2005, The Journal of Neuroscience.

[13]  K. Hiraki,et al.  Rubber Hand Illusion under Delayed Visual Feedback , 2009, PloS one.

[14]  Noriaki Kanayama,et al.  Multisensory processing and neural oscillatory responses: separation of visuotactile congruency effect and corresponding electroencephalogram activities , 2009, Neuroreport.

[15]  Yuji Ishino,et al.  A simple action of right index finger induces rubber hand illusion to static left hand , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[16]  Julian Keil,et al.  Localization of the magnetic equivalent of the ERN and induced oscillatory brain activity , 2010, NeuroImage.

[17]  W. Singer Consciousness and the Binding Problem , 2001, Annals of the New York Academy of Sciences.

[18]  D. Tucker,et al.  Frontal midline theta and the error-related negativity: neurophysiological mechanisms of action regulation , 2004, Clinical Neurophysiology.

[19]  Arnaud Delorme,et al.  EEGLAB, SIFT, NFT, BCILAB, and ERICA: New Tools for Advanced EEG Processing , 2011, Comput. Intell. Neurosci..

[20]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[21]  John J. B. Allen,et al.  Theta lingua franca: a common mid-frontal substrate for action monitoring processes. , 2012, Psychophysiology.

[22]  V. Gallese,et al.  Upcoming tactile events and body ownership in schizophrenia , 2014, Schizophrenia Research.

[23]  C. Spence,et al.  Spatial constraints on visual-tactile cross-modal distractor congruency effects , 2004, Cognitive, affective & behavioral neuroscience.

[24]  Wataru Teramoto,et al.  Visuotactile interaction even in far sagittal space in older adults with decreased gait and balance functions , 2017, Experimental Brain Research.

[25]  T. Metzinger,et al.  Video Ergo Sum: Manipulating Bodily Self-Consciousness , 2007, Science.

[26]  Rafal Bogacz,et al.  Theta phase resetting and the error-related negativity. , 2007, Psychophysiology.

[27]  Noriaki Kanayama,et al.  BDNF as a possible modulator of EEG oscillatory response at the parietal cortex during visuo-tactile integration processes using a rubber hand , 2017, Neuroscience Research.

[28]  M. Berger,et al.  High Gamma Power Is Phase-Locked to Theta Oscillations in Human Neocortex , 2006, Science.

[29]  John J. B. Allen,et al.  Theta EEG dynamics of the error-related negativity , 2007, Clinical Neurophysiology.

[30]  C. Spence,et al.  Visual Capture of Touch: Out-of-the-Body Experiences With Rubber Gloves , 2000, Psychological science.

[31]  Kazuo Hiraki,et al.  Infant's brain responses to live and televised action , 2006, NeuroImage.

[32]  Noriaki Kanayama,et al.  The role of gamma band oscillations and synchrony on rubber hand illusion and crossmodal integration , 2009, Brain and Cognition.

[33]  Georg Northoff,et al.  Temporal limits on rubber hand illusion reflect individuals’ temporal resolution in multisensory perception , 2016, Cognition.

[34]  Wolf Singer,et al.  Neuronal Synchrony: A Versatile Code for the Definition of Relations? , 1999, Neuron.