Audio in VR: Effects of a Soundscape and Movement-Triggered Step Sounds on Presence

For effective virtual realities, “presence,” the feeling of “being there” in a virtual environment (VR), is deemed an essential prerequisite. Several studies have assessed the effect of the (non-)availability of auditory stimulation on presence, but due to differences in study design (e.g., virtual realities used, types of sounds included, rendering technologies employed), generalizing the results and estimating the effect of the auditory component is difficult. In two experiments, the influence of an ambient nature soundscape and movement-triggered step sounds were investigated regarding their effects on presence. In each experiment, approximately forty participants walked on a treadmill, thereby strolling through a virtual park environment reproduced via a stereoscopic head-mounted display (HMD), while the acoustical environment was delivered via noise-canceling headphones. In Experiment 1, conditions with the ambient soundscape and the step sounds either present or absent were combined in a 2 × 2 within-subjects design, supplemented with an additional “no-headphones” control condition. For the synchronous playback of step sounds, the probability of a step being taken was estimated by an algorithm using the HMD’s sensor data. The results of Experiment 1 show that questionnaire-based measures of presence and realism were influenced by the soundscape but not by the reproduction of steps, which might be confounded with the fact that the perceived synchronicity of the sensor-triggered step sounds was rated rather low. Therefore, in Experiment 2, the step-reproduction algorithm was improved and judged to be more synchronous by participants. Consequently, large and statistically significant effects of both kinds of audio manipulations on perceived presence and realism were observed, with the effect of the soundscape being larger than that of including footstep sounds, possibly due to the remaining imperfections in the reproduction of steps. Including an appropriate soundscape or self-triggered footsteps had differential effects on subscales of presence, in that both affected overall presence and realism, while involvement was improved and distraction reduced by the ambient soundscape only.

[1]  Henrik Møller,et al.  Binaural Technique: Basic Methods for Recording, Synthesis, and Reproduction , 2005 .

[2]  Torsten Dau,et al.  Sound source localization with varying amount of visual information in virtual reality , 2018, bioRxiv.

[3]  Luca Turchet,et al.  Sound Synthesis and Evaluation of Interactive Footsteps and Environmental Sounds Rendering for Virtual Reality Applications , 2011, IEEE Transactions on Visualization and Computer Graphics.

[4]  Wijnand A. IJsselsteijn,et al.  Presence: concept, determinants, and measurement , 2000, Electronic Imaging.

[5]  Russell F. Pinkston,et al.  A Touch Sensitive Dance Floor/MIDI Controller , 1995, ICMC.

[6]  A.W. Law,et al.  A multi-modal floor-space for experiencing material deformation underfoot in virtual reality , 2008, 2008 IEEE International Workshop on Haptic Audio visual Environments and Games.

[7]  Woodrow Barfield,et al.  The Sense of Presence within Auditory Virtual Environments , 1996, Presence: Teleoperators & Virtual Environments.

[8]  Maria V. Sanchez-Vives,et al.  How we experience immersive virtual environments: the concept of presence and its measurement * , 2009 .

[9]  Vincent Hayward,et al.  Preliminary Experiment Combining Virtual Reality Haptic Shoes and Audio Synthesis , 2010, EuroHaptics.

[10]  Mel Slater,et al.  A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments , 1997, Presence: Teleoperators & Virtual Environments.

[11]  Mel Slater,et al.  Taking steps: the influence of a walking technique on presence in virtual reality , 1995, TCHI.

[12]  Deep Sen,et al.  Efficient, Compelling, and Immersive VR Audio Experience Using Scene Based Audio/Higher Order Ambisonics , 2016 .

[13]  Joseph A. Paradiso,et al.  The magic carpet: physical sensing for immersive environments , 1997, CHI Extended Abstracts.

[14]  Holger Regenbrecht,et al.  The Experience of Presence: Factor Analytic Insights , 2001, Presence: Teleoperators & Virtual Environments.

[15]  Mendel Kleiner,et al.  Auditory-Induced Presence in Mixed Reality Environments and Related Technology , 2010, The Engineering of Mixed Reality Systems.

[16]  Daniel Västfjäll,et al.  The Actor-Observer Effect in Virtual Reality Presentations , 2001, Cyberpsychology Behav. Soc. Netw..

[17]  Frank Biocca,et al.  The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments , 2003, Presence: Teleoperators & Virtual Environments.

[18]  Wolfgang Ellermeier,et al.  Noise-cancelling, steps and soundscapes: the effect of auditory stimulation on presence in virtual realities while walking , 2016, VRST.

[19]  Jeremy N. Bailenson,et al.  How Immersive Is Enough? A Meta-Analysis of the Effect of Immersive Technology on User Presence , 2016 .

[20]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[21]  T. Kuyel,et al.  Real‐time computer simulation of concert hall acoustics: Arbitrary geometries , 1994 .

[22]  Michael P. Snow,et al.  Empirical Models Based on Free-Modulus Magnitude Estimation of Perceived Presence in Virtual Environments , 1998, Hum. Factors.

[23]  Stefania Serafin,et al.  Sound design and perception in walking interactions , 2009, Int. J. Hum. Comput. Stud..

[24]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[25]  Oskar von Stryk,et al.  Real-time step detection using the integrated sensors of a head-mounted display , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).