Walking in the Head: Methods of Sonic Augmented Reality Navigation

This paper describes an experimental method for the navigation of immersive sonic augmented reality experiences. The focus is on tight integration of the audio experience with the real environment in conjunction with high quality immersion in both the real space as well as the augmented information presented throughout the experience. Drawing from lessons learned from two projects that present narrative content pertaining to the historic layers of two places in the form of location-specific immersive audio experiences embedded in the current, real environment, we discuss techniques of sonic immersion and auditory spatial navigation strategies. With custom-developed headsets the realization and performance of bone-conducted spatial sound is explored and design considerations for such experiences are formulated.

[1]  Rolf Nordahl,et al.  Missing the point: an exploration of how to guide users' attention during cinematic virtual reality , 2016, VRST.

[2]  Stefania Serafin,et al.  Binaural sound reduces reaction time in a virtual reality search task , 2017, 2017 IEEE 3rd VR Workshop on Sonic Interactions for Virtual Environments (SIVE).

[3]  Woon-Seng Gan,et al.  Natural Listening over Headphones in Augmented Reality Using Adaptive Filtering Techniques , 2015, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[4]  Brian F. G. Katz,et al.  Impact of HRTF Individualization on Player Performance in a VR Shooter Game I , 2018 .

[5]  Michael J. Gerzon.  Periphony: With-Height Sound Reproduction , 1973 .

[6]  Fumihisa Shibata,et al.  A Two-by-Two Mixed Reality System That Merges Real and Virtual Worlds in Both Audio and Visual Senses , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[7]  Keiichi Zempo,et al.  Measurement of the differential transfer function between bone-conduction and air-conduction for sound localization , 2015, 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE).

[8]  George Drettakis,et al.  Accommodation and Comfort in Head-Mounted Displays , 2018 .

[9]  Woon-Seng Gan,et al.  Fast HRFT measurement system with unconstrained head movements for 3D audio in virtual and augmented reality applications , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Naotaka Fujii,et al.  Substitutional Reality System: A Novel Experimental Platform for Experiencing Alternative Reality , 2012, Scientific Reports.

[11]  Karla Berrens.  An emotional cartography of resonance , 2016 .

[12]  Tapio Lokki,et al.  Application Scenarios of Wearable and Mobile Augmented Reality Audio , 2004 .

[13]  Paula P. Henry,et al.  Spatial audio through a bone conduction interface , 2006, International journal of audiology.

[14]  M. Pérez,et al.  Contribution of Tau Pathology to Mitochondrial Impairment in Neurodegeneration , 2018, Front. Neurosci..

[15]  S. Helmreich.  Listening Against Soundscapes , 2010 .

[16]  Robert W. Lindeman,et al.  An Empirical Study of Hear-Through Augmented Reality: Using Bone Conduction to Deliver Spatialized Audio , 2008, 2008 IEEE Virtual Reality Conference.

[17]  A. Eiber,et al.  Sound wave propagation on the human skull surface with bone conduction stimulation , 2017, Hearing Research.

[18]  Dubravko Miljkovic,et al.  Active noise control: From analog to digital - Last 80 years , 2016, MIPRO.

[19]  Mark Billinghurst,et al.  Spatial sound localization in an augmented reality environment , 2006, OZCHI.

[20]  Mamoru Iwaki,et al.  Compensation of sound source direction perceived through consumer-grade bone-conduction headphones by modifying ILD and ITD , 2016, 2016 IEEE 5th Global Conference on Consumer Electronics.

[21]  Hans-Werner Gellersen,et al.  Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences , 2015, CHI.

[22]  Keiichi Zempo,et al.  Analysis of sound propagation in human head for bone-conduction headphones using finite element method , 2014, 2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE).

[23]  Wen Zhang,et al.  Surround by Sound: A Review of Spatial Audio Recording and Reproduction , 2017 .

[24]  Dimitar Valkov,et al.  Smooth immersion: the benefits of making the transition to virtual environments a continuous process , 2017, SUI.

[25]  Aljoscha Smolic,et al.  Director's cut: a combined dataset for visual attention analysis in cinematic VR content , 2018, CVMP '18.

[26]  Gregory Kramida,et al.  Resolving the Vergence-Accommodation Conflict in Head-Mounted Displays , 2016, IEEE Transactions on Visualization and Computer Graphics.

[27]  Kazuhiro Kondo,et al.  Characteristics comparison of two audio output devices for augmented audio reality , 2013, 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.

[28]  Ana Tajadura-Jiménez,et al.  Generic HRTFs May be Good Enough in Virtual Reality. Improving Source Localization through Cross-Modal Plasticity , 2018, Front. Neurosci..