Virtually the same? How impaired sensory information in virtual reality may disrupt vision for action

Virtual reality (VR) is a promising tool for expanding the possibilities of psychological experimentation and implementing immersive training applications. Despite a recent surge in interest, there remains an inadequate understanding of how VR impacts basic cognitive processes. Due to the artificial presentation of egocentric distance cues in virtual environments, a number of cues to depth in the optic array are impaired or placed in conflict with each other. Moreover, realistic haptic information is all but absent from current VR systems. The resulting conflicts could impact not only the execution of motor skills in VR but also raise deeper concerns about basic visual processing, and the extent to which virtual objects elicit neural and behavioural responses representative of real objects. In this brief review, we outline how the novel perceptual environment of VR may affect vision for action, by shifting users away from a dorsal mode of control. Fewer binocular cues to depth, conflicting depth information and limited haptic feedback may all impair the specialised, efficient, online control of action characteristic of the dorsal stream. A shift from dorsal to ventral control of action may create a fundamental disparity between virtual and real-world skills that has important consequences for how we understand perception and action in the virtual world.

[1]  A S Eadie,et al.  Modelling adaptation effects in vergence and accommodation after exposure to a simulated Virtual Reality stimulus , 2000, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[2]  Douglas Lanman,et al.  Fast gaze-contingent optimal decompositions for multifocal displays , 2017, ACM Trans. Graph..

[3]  Heinrich H. Bülthoff,et al.  Virtual arm׳s reach influences perceived distances but only after experience reaching , 2015, Neuropsychologia.

[4]  Darrin O Wijeyaratnam,et al.  Going offline: differences in the contributions of movement control processes when reaching in a typical versus novel environment , 2019, Experimental Brain Research.

[5]  Mark Mon-Williams,et al.  Natural problems for stereoscopic depth perception in virtual environments , 1995, Vision Research.

[6]  Gerd Bruder,et al.  Virtual proxemics: Locomotion in the presence of obstacles in large immersive projection environments , 2015, 2015 IEEE Virtual Reality (VR).

[7]  G. Buckingham,et al.  Reframing the action and perception dissociation in DF: haptics matters, but how? , 2013, Journal of neurophysiology.

[8]  Heinrich H. Bülthoff,et al.  The Effect of Viewing a Self-Avatar on Distance Judgments in an HMD-Based Virtual Environment , 2010, PRESENCE: Teleoperators and Virtual Environments.

[9]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[10]  Anthony Steed,et al.  Individual Differences in Embodied Distance Estimation in Virtual Reality , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[11]  Jacqueline C. Snow,et al.  Priming tool actions: Are real objects more effective primes than pictures? , 2015, Experimental Brain Research.

[12]  Jacqueline C Snow,et al.  Action Properties of Object Images Facilitate Visual Search , 2017, Journal of experimental psychology. Human perception and performance.

[13]  Melvyn A. Goodale,et al.  Real-time vision, tactile cues, and visual form agnosia: removing haptic feedback from a “natural” grasping task induces pantomime-like grasps , 2015, Front. Hum. Neurosci..

[14]  Jonathan S. Cant,et al.  Coming to grips with vision and touch , 2007, Behavioral and Brain Sciences.

[15]  Gavin Buckingham,et al.  Examining the size–weight illusion with visuo-haptic conflict in immersive virtual reality , 2019, Quarterly journal of experimental psychology.

[16]  M. Mon-Williams,et al.  When two eyes are better than one in prehension: monocular viewing and end-point variance , 2004, Experimental Brain Research.

[17]  M. Goodale,et al.  Does a monocularly presented size-contrast illusion influence grip aperture? , 1998, Neuropsychologia.

[18]  E. Tunik,et al.  Sensorimotor training in virtual reality: a review. , 2009, NeuroRehabilitation.

[19]  Philippe Coiffet,et al.  Virtual Reality Technology , 2003, Presence: Teleoperators & Virtual Environments.

[20]  M. Mon-Williams,et al.  Some Recent Studies on the Extraretinal Contribution to Distance Perception , 1999, Perception.

[21]  Changwon Jang,et al.  Retinal 3D , 2017, ACM Trans. Graph..

[22]  M. Goodale,et al.  Chapter 28 Visual pathways to perception and action , 1993 .

[23]  Victoria Interrante,et al.  Distance Perception in Immersive Virtual Environments, Revisited , 2006, IEEE Virtual Reality Conference (VR 2006).

[24]  Markus Broecker,et al.  Bogus Visual Feedback Alters Onset of Movement-Evoked Pain in People With Neck Pain , 2015, Psychological science.

[25]  Gereon R Fink,et al.  Are action and perception in near and far space additive or interactive factors? , 2003, NeuroImage.

[26]  Victoria Interrante,et al.  An experimental investigation of distance perception in real vs. immersive virtual environments via direct blind walking in a high-fidelity model of the same room , 2004, APGV '04.

[27]  Michael Meehan,et al.  Physiological measures of presence in stressful virtual environments , 2002, SIGGRAPH.

[28]  Mar Gonzalez-Franco,et al.  The uncanny valley of haptics , 2018, Science Robotics.

[29]  Torsten Kuhlen,et al.  Evaluation of Spatial Processing in Virtual Reality Using Functional Magnetic Resonance Imaging (fMRI) , 2009, Cyberpsychology Behav. Soc. Netw..

[30]  Jacqueline C. Snow,et al.  Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions , 2019, Cortex.

[31]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[32]  James R. Tresilian,et al.  Increasing confidence in vergence as a cue to distance , 1999, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[33]  James M. Finley,et al.  The quality of visual information about the lower extremities influences visuomotor coordination during virtual obstacle negotiation. , 2018, Journal of neurophysiology.

[34]  M. Goodale,et al.  Visual pathways to perception and action. , 1993, Progress in brain research.

[35]  James R. Tresilian,et al.  Monocular and binocular distance cues: insights from visual form agnosia I (of III) , 2001, Experimental Brain Research.

[36]  Giuseppe Riva,et al.  Virtual Reality Body Swapping: A Tool for Modifying the Allocentric Memory of the Body , 2016, Cyberpsychology Behav. Soc. Netw..

[37]  M. Goodale,et al.  Visual control of action but not perception requires analytical processing of object shape , 2003, Nature.

[38]  L. Jakobson,et al.  Differences in the visual control of pantomimed and natural grasping movements , 1994, Neuropsychologia.

[39]  Rob Gray,et al.  Virtual environments and their role in developing perceptual-cognitive skills in sports , 2019, Anticipation and Decision Making in Sport.

[40]  W. Geoffrey Wright,et al.  Using virtual reality to augment perception, enhance sensorimotor adaptation, and change our minds , 2014, Front. Syst. Neurosci..

[41]  R. Aggarwal,et al.  Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery , 2008, The British journal of surgery.

[42]  A. Milner,et al.  Perception and Action in Depth , 1998, Consciousness and Cognition.

[43]  M. Mon-Williams,et al.  Binocular vision in a virtual world: visual deficits following the wearing of a head‐mounted display , 1993, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.