Virtual environments with soundscapes: a study on immersion and effects of spatial abilities

In this study we explore how soundscapes can be used as navigational aids in virtual environments and empirically investigate the correlation between immersiveness in virtual environments and spatial abilities when soundscapes are used as landmarks for wayfinding. We attempt to advance knowledge regarding auditory cues contributing to enhanced immersive and navigational experience in virtual environments. Findings are likely to be utilized in effective design for physical environments and wayfinding.

[1]  Vassil N. Alexandrov,et al.  Use of auditory cues for wayfinding assistance in virtual environment: music aids route decision , 2008, VRST '08.

[2]  R. Golledge,et al.  Spatial Behavior: A Geographic Perspective , 1996 .

[3]  Christopher F. Chabris,et al.  Prelude or requiem for the ‘Mozart effect’? , 1999, Nature.

[4]  Kevin Lynch,et al.  The Image of the City , 1960 .

[5]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[6]  Frans Mäyrä,et al.  Fundamental Components of the Gameplay Experience: Analysing Immersion , 2005, DiGRA Conference.

[7]  H. Stumpf,et al.  A structural analysis of visual spatial ability in academically talented students , 1999 .

[8]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[9]  Hanspeter A. Mallot,et al.  The Role of Global and Local Landmarks in Virtual Environment Navigation , 2000, Presence Teleoperators Virtual Environ..

[10]  Andreas Komninos,et al.  Urban exploration using audio scents , 2012, Mobile HCI.

[11]  Lotfi B. Merabet,et al.  Audio-Based Navigation Using Virtual Environments: Combining Technology and Neuroscience , 2009 .

[12]  Mary Czerwinski,et al.  Immersion in desktop virtual reality , 1997, UIST '97.

[13]  Chris Dede,et al.  Immersive Interfaces for Engagement and Learning , 2009, Science.

[14]  Kathy Pezdek,et al.  Cognitive Maps and Urban Form , 1982 .

[15]  M. Hegarty,et al.  A dissociation between mental rotation and perspective-taking spatial abilities , 2004 .

[16]  Stephen A. Brewster,et al.  Using nonspeech sounds to provide navigation cues , 1998, TCHI.

[17]  D. Hall,et al.  Perception of soundscapes: An interdisciplinary approach , 2013 .

[18]  Michael J. Singer,et al.  On Selecting the Right Yardstick , 1999, Presence.

[19]  Michael C. Pyryt.  Human cognitive abilities: A survey of factor analytic studies , 1998 .

[20]  Steven D. Pieper,et al.  Hands-on interaction with virtual environments , 1989, UIST '89.

[21]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[22]  David B. Kaber,et al.  Telepresence , 1998, Hum. Factors.

[23]  Daniel Västfjäll,et al.  Auditory Landmarks Enhance Circular Vection in Multimodal Virtual Reality , 2009 .

[24]  S. Kaufman.  Sex differences in mental rotation and spatial visualization ability: Can they be accounted for by differences in working memory capacity? , 2007 .

[25]  Maria V. Sanchez-Vives,et al.  From presence to consciousness through virtual reality , 2005, Nature Reviews Neuroscience.

[26]  Lois Hetland,et al.  Listening to Music Enhances Spatial-Temporal Reasoning: Evidence for the "Mozart Effect.". , 2000 .

[27]  Kensuke Shimizu,et al.  "Back-Seat Driver": Spatial Sound for Vehicular Way-Finding and Situation Awareness , 2006, 2006 Japan-China Joint Workshop on Frontier of Computer Science and Technology.

[28]  E. Schellenberg,et al.  Arousal, Mood, and The Mozart Effect , 2001, Psychological science.

[29]  Manfred Tscheligi,et al.  CHI '04 Extended Abstracts on Human Factors in Computing Systems , 2004, CHI 2004.

[30]  George Drettakis,et al.  Perceptual audio rendering of complex virtual environments , 2004, ACM Trans. Graph..

[31]  M. G. McGee.  Human spatial abilities: psychometric studies and environmental, genetic, hormonal, and neurological influences. , 1979, Psychological bulletin.

[32]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[33]  Peter Dixon,et al.  A limiting feature of the Mozart effect: listening enhances mental rotation abilities in non-musicians but not musicians , 2010 .

[34]  Jason Brandon Abrams,et al.  Wayfinding in Architecture , 2010 .

[35]  Markus H. Gross,et al.  Spatialized audio rendering for immersive virtual environments , 2002, VRST '02.

[36]  Laurie Radford.  Barry Truax, editor: Handbook of Acoustic Ecology, 2nd edition (CD-ROM version) [Review Article] , 2001 .

[37]  Woodrow Barfield,et al.  Presence within Virtual Environments as a Function of Visual Display Parameters , 1996, Presence: Teleoperators & Virtual Environments.

[38]  F. O. Huck,et al.  Visual Information Processing , 1992 .

[39]  Kristine Jørgensen,et al.  On the Functional Aspects of Computer Game Audio , 2006 .

[40]  David A. Rettinger,et al.  How are visuospatial working memory, executive functioning, and spatial abilities related? A latent-variable analysis. , 2001, Journal of experimental psychology. General.

[41]  Camille Goudeseune,et al.  Model based interactive sound for an immersive virtual environment , 1994, ICMC.

[42]  Mel Slater,et al.  Depth of Presence in Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[43]  Blake Hannaford,et al.  Stable haptic interaction with virtual environments , 1999, IEEE Trans. Robotics Autom..

[44]  Rick Kazman,et al.  Using 3D sound as a navigational aid in virtual environments , 2004, Behav. Inf. Technol..

[45]  Ronald R. Mourant,et al.  Effect of Visual Display Parameters on Driving Performance in a Virtual Environments Driving Simulator , 1996 .

[46]  David C. Geary,et al.  Sexual selection and sex differences in mathematical abilities , 1996, Behavioral and Brain Sciences.

[47]  C. Adcock,et al.  Primary Mental Abilities. , 1971, The Journal of general psychology.

[48]  Danièle Dubois,et al.  CATEGORIES FOR SOUNDSCAPE : TOWARD A HYBRID CLASSIFICATION , 2010 .

[49]  Bruce N. Walker,et al.  Effect of Beacon Sounds on Navigation Performance in a Virtual Reality Environment , 2003 .

[50]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[51]  Roberta L. Klatzky,et al.  Navigation System for the Blind: Auditory Display Modes and Guidance , 1998, Presence.

[52]  Patrick Susini,et al.  Soundscape design in train stations : Perceptual study of soundscapes , 2009 .

[53]  E. Maguire,et al.  Knowing Where Things Are: Parahippocampal Involvement in Encoding Object Locations in Virtual Large-Scale Space , 1998, Journal of Cognitive Neuroscience.

[54]  Rosa María Baños,et al.  Immersion and Emotion: Their Impact on the Sense of Presence , 2004, Cyberpsychology Behav. Soc. Netw..

[55]  Knut Hartmann,et al.  Composition and Arrangement Techniques for Music in Interactive Immersive Environments , 2006 .

[56]  Norman G. Vinson,et al.  Design guidelines for landmarks to support navigation in virtual environments , 1999, CHI '99.

[57]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[58]  F. Rauscher,et al.  Music and spatial task performance , 1993, Nature.

[59]  S. Shyam Sundar,et al.  Where Am I? How Can I Get There? Impact of Navigability and Narrative Transportation on Spatial Presence , 2011, Hum. Comput. Interact..

[60]  Jian Kang,et al.  Towards the Evaluation, Description, and Creation of Soundscapes in Urban Open Spaces , 2007 .

[61]  Jordi Janer,et al.  Soundscape Generation for Virtual Environments using Community-Provided Audio Databases , 2010 .

[62]  K. Kallinen,et al.  Presence and emotion in computer game players during 1st person vs. 3rd person playing view: evidence from self-report, eye-tracking, and facial muscle activity data , 2007 .

[63]  Robert F. DeVellis,et al.  Scale Development: Theory and Applications. , 1992 .