Using Audio Reverberation to Compensate Distance Compression in Virtual Reality

Virtual Reality (VR) technologies are increasingly being applied to various contexts like those gaming, therapy, training, and education. Several of these applications require high degrees of accuracy in spatial and depth perception. Contemporary VR experiences continue to be confronted by the issue of distance compression wherein people systematically underestimate distances in the virtual world, leading to impoverished experiences. Consequently, a number of studies have focused on extensively understanding and exploring factors that influence this phenomenon to address the challenges it poses. Inspired by previous work that has sought to compensate distance compression effects in VR, we examined the potential of manipulating an auditory stimulus’ reverberation time (RT) to alter how users perceive depth. To this end, we conducted a two action forced choice study in which participants repeatedly made relative depth judgements between a pair of scenes featuring a virtual character placed at different distances with varying RTs. Results revealed that RT influences how users perceive depth with this influence being more pronounced in the near field. We found that users tend to associate longer RTs with farther distances and vice versa, indicating the potential to alter RT towards compensating distance underestimation in VR. However, it must be noted that excessively increasing RT (especially in the near field) could come at the cost of sensory segregation, wherein users are unable to unify visual and auditory sensory stimuli in their perceptions of depth. Researchers must hence strive to find the optimal amount of RT to add to a stimulus to ensure seamless virtual experiences.

[1]  Victoria Interrante,et al.  Distance Perception in Immersive Virtual Environments, Revisited , 2006, IEEE Virtual Reality Conference (VR 2006).

[2]  Sabarish V. Babu,et al.  iVRNote: Design, Creation and Evaluation of an Interactive Note-Taking Interface for Study and Reflection in VR Learning Environments , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[3]  B. Fazenda,et al.  The effect of reverberation and audio spatialization on egocentric distance estimation of objects in stereoscopic virtual reality , 2017 .

[4]  Gerd Bruder,et al.  Visual blur in immersive virtual environments: does depth of field or motion blur affect distance and speed estimation? , 2016, VRST.

[5]  Henry Y. K. Lau,et al.  Corrective feedback for depth perception in CAVE-like systems , 2017, 2017 IEEE Virtual Reality (VR).

[6]  Boris M. Velichkovsky,et al.  The perception of egocentric distances in virtual environments - A review , 2013, ACM Comput. Surv..

[7]  Frank H. Durgin,et al.  Distance Perception and the Visual Horizon in Head-Mounted Displays , 2005, TAP.

[8]  Eamonn O'Neill,et al.  Compensating for Distance Compression in Audiovisual Virtual Environments Using Incongruence , 2016, CHI.

[9]  Heinrich H. Bülthoff,et al.  Eye Height Manipulations: A Possible Solution to Reduce Underestimation of Egocentric Distances in Head-Mounted Displays , 2015, TAP.

[10]  Damian Murphy,et al.  OpenAIR: An Interactive Auralization Web Resource and Database , 2010 .

[11]  J. Wald,et al.  Efficacy of virtual reality exposure therapy to treat driving phobia: a case report. , 2000, Journal of behavior therapy and experimental psychiatry.

[12]  D. Mershon,et al.  Intensity and reverberation as factors in the auditory perception of egocentric distance , 1975 .

[13]  Kevin Ponto,et al.  Perceptual space warping: Preliminary exploration , 2016, 2016 IEEE Virtual Reality (VR).

[14]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[15]  Bobby Bodenheimer,et al.  Distance estimation in virtual environments using different HMDs , 2014, SAP.

[16]  Peter Willemsen,et al.  Throwing versus walking as indicators of distance perception in similar real and virtual environments , 2005, TAP.

[17]  L. Shams,et al.  Crossmodal influences on visual perception. , 2010, Physics of life reviews.

[18]  Allan J. Zuckerwar,et al.  Acoustics: Sound Fields and Transducers , 1954 .

[19]  Pablo E Etchemendy,et al.  Auditory environmental context affects visual distance perception , 2017, Scientific Reports.

[20]  Henk G. Stassen,et al.  Theoretical background and conceptual solution for depth perception and eye-hand coordination problems in laparoscopic surgery , 1999 .

[21]  Adrian David Cheok,et al.  An experimental study on the role of software synthesized 3D sound in augmented reality environments , 2004, Interact. Comput..

[22]  Jannick P. Rolland,et al.  Towards Quantifying Depth and Size Perception in Virtual Environments , 1993, Presence: Teleoperators & Virtual Environments.

[23]  Hao-Chuan Wang,et al.  Using Spatialized Audio to Improve Human Spatial Knowledge Acquisition in Virtual Reality , 2018, IUI Companion.

[24]  Jack M. Loomis,et al.  Visual perception of egocentric distance in real and virtual environments. , 2003 .

[25]  Qi Li,et al.  The Bayesian Causal Inference in Multisensory Information Processing: A Narrative Review , 2018, IIH-MSP.

[26]  D L Fisher,et al.  Can novice drivers be trained to scan for information that will reduce their likelihood of a crash? , 2006, Injury Prevention.

[27]  Hyun Joon Shin,et al.  Perceptual Calibration for Immersive Display Environments , 2013, IEEE Transactions on Visualization and Computer Graphics.

[28]  Phillip M. Harter,et al.  Simulation for Team Training and Assessment: Case Studies of Online Training with Virtual Worlds , 2008, World Journal of Surgery.

[29]  Vesa Välimäki,et al.  Fifty Years of Artificial Reverberation , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[30]  Nick Holliman,et al.  Can the perception of depth in stereoscopic images be influenced by 3D sound? , 2011, Electronic Imaging.

[31]  Chiuhsiang Joe Lin,et al.  Distance estimation with mixed real and virtual targets in stereoscopic displays , 2015, Displays.

[32]  Christine J. Ziemer,et al.  Estimating distance in real and virtual environments: Does order make a difference? , 2009, Attention, perception & psychophysics.

[33]  Frank Melchior,et al.  Object-Based Reverberation for Spatial Audio , 2017 .

[34]  Andrew Hogue,et al.  Sound and stereoscopic 3D: Examining the effects of sound on depth perception in stereoscopic 3D , 2016, 2016 7th International Conference on Information, Intelligence, Systems & Applications (IISA).

[35]  Bobby Bodenheimer,et al.  A Comparison of Distance Estimation in HMD-Based Virtual Environments with Different HMD-Based Conditions , 2018, ACM Trans. Appl. Percept..

[36]  Gerd Bruder,et al.  Distance estimation in large immersive projection systems, revisited , 2015, 2015 IEEE Virtual Reality (VR).

[37]  Bochao Li,et al.  Minication affects action-based distance judgments in oculus rift HMDs , 2014, SAP.

[38]  Sabarish V. Babu,et al.  Empirically Evaluating the Effects of Perceptual Information Channels on the Size Perception of Tangibles in Near-Field Virtual Reality , 2021, 2021 IEEE Virtual Reality and 3D User Interfaces (VR).

[39]  Stephanie C. Boyle Investigating the neural mechanisms underlying audio-visual perception using electroencephalography (EEG) , 2018 .

[40]  Tien Dat Nguyen,et al.  How does presentation method and measurement protocol affect distance estimation in real and virtual environments? , 2010, TAP.

[41]  Jonathan W. Kelly,et al.  Comparison of Two Methods for Improving Distance Perception in Virtual Reality , 2018, ACM Trans. Appl. Percept..

[42]  Marina-Anca Cidotã,et al.  Using Visual Effects to Facilitate Depth Perception for Spatial Tasks in Virtual and Augmented Reality , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).

[43]  Christopher C. Pagano,et al.  Investigating the Effects of Anthropomorphic Fidelity of Self-Avatars on Near Field Depth Perception in Immersive Virtual Environments , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[44]  Fernando Iazzettaa,et al.  AcMus : Computational Tools for Measurement , Analysis and Simulation of Room Acoustics , 2004 .

[45]  Ekaterina P. Volkova,et al.  Emotion categorization of body expressions in narrative scenarios , 2014, Front. Psychol..

[46]  Bobby Bodenheimer,et al.  A comparison of two cost-differentiated virtual reality systems for perception and action tasks , 2014, SAP.

[47]  Bob G. Witmer,et al.  Judging Perceived and Traversed Distance in Virtual Environments , 1998, Presence.

[48]  Peng Liu,et al.  Impact of visual and experiential realism on distance perception in VR using a custom video see-through system , 2017, SAP.

[49]  Nicolas S. Holliman,et al.  3D sound and 3D image interactions: a review of audio-visual depth perception , 2014, Electronic Imaging.

[50]  Bliss M. Altenhoff,et al.  Carryover effects of calibration to visual and proprioceptive information on near field distance judgments in 3D user interaction , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[51]  Tammo Houtgast,et al.  Auditory distance perception in rooms , 1999, Nature.

[52]  Peter Willemsen,et al.  The effects of head-mounted display mechanical properties and field of view on distance judgments in virtual environments , 2009, TAP.

[53]  Neofytos Kaplanis,et al.  Perception of reverberation in small rooms : a literature study , 2014 .

[54]  William B. Thompson,et al.  Minification influences spatial judgments in virtual environments , 2006, APGV '06.

[55]  Heinrich H. Bülthoff,et al.  The MPI Emotional Body Expressions Database for Narrative Scenarios , 2014, PloS one.

[56]  Daniel J. Finnegan,et al.  Compensating for distance compression in virtual audiovisual environments , 2017 .

[57]  Adam R. Richardson,et al.  The effect of feedback training on distance estimation in virtual environments , 2005 .

[58]  Uwe Kloos,et al.  The influence of eye height and avatars on egocentric distance estimates in immersive virtual environments , 2011, APGV '11.