Human Factors Research in Audio Augmented Reality

Within the burgeoning field of augmented reality (AR), audio AR is a medium whereby 2D or 3D spatial audio is registered to the user’s surroundings, using their position and head-orientation. In locative audio applications, such as several that have been released for smart-phones, audio is presented only in relationship to the user’s location. Audio AR that uses both these sensor inputs has the advantage that both the semantic and directional information content of the aural presentation may be perceived without paying attention to the hardware device. In comparison to visual AR, which often requires engaging with a screen on a mobile device, audio AR enables hands-free, passive reception of AR content. Human spatial hearing allows awareness of the direction and distance of virtual sonic objects throughout the full sphere, in comparison to the limited momentary field of view for visual AR. This chapter situates the author’s research within a wider survey of human factors research in audio AR systems. It provides a taxonomy of audio AR systems, and a framework for considering human factors in terms of users’ abilities and affordances of the systems. Research is reviewed that examines the perceptual and task-related performance shortfalls between ideal audio AR and what is achievable with practical, current technology.

[1]  Mark A. Livingston,et al.  Evaluating Human Factors in Augmented Reality Systems , 2005, IEEE Computer Graphics and Applications.

[2]  Elizabeth D. Mynatt,et al.  Designing audio aura , 1998, CHI.

[3]  William L. Martens,et al.  Perceptual evaluation of filters controlling source direction : Customized and generalized HRTFs for binaural synthesis , 2003 .

[4]  S. Carlile,et al.  Discrimination of sound source velocity in human listeners. , 2002, The Journal of the Acoustical Society of America.

[5]  Richard L. McKinley,et al.  The Interaction Between Head-Tracker Latency, Source Duration, and Response Time in the Localization of Virtual Sound Sources , 2004, ICAD.

[6]  Michael Cohen,et al.  Augmented audio reality: telepresence/VR hybrid acoustic environments , 1993, Proceedings of 1993 2nd IEEE International Workshop on Robot and Human Communication.

[7]  Nicholas Mariette.  Navigation Performance Effects of Render Method and Head-Turn Latency in Mobile Audio Augmented Reality , 2009, CMMR/ICAD.

[8]  Pedro Novo,et al.  Auditory Virtual Environments , 2005 .

[9]  Adrian David Cheok,et al.  The Role of 3-D Sound in Human Reaction and Performance in Augmented Reality Environments , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[10]  Juan Carlos Augusto,et al.  Handbook of Ambient Intelligence and Smart Environments , 2009 .

[11]  Kent Lyons,et al.  Guided by voices: An audio augmented reality system , 2000 .

[12]  André van Schaik,et al.  Psychoacoustic Evaluation of Systems for Delivering Spatialized Augmented-Reality Audio , 2009 .

[13]  Bruce N. Walker,et al.  Navigation Performance With a Virtual Auditory Display: Effects of Beacon Sound, Capture Radius, and Practice , 2006, Hum. Factors.

[14]  A. John Van Opstal,et al.  Plasticity in human sound localization induced by compressed spatial vision , 2003, Nature Neuroscience.

[15]  J. J. Gibson.  The theory of affordances , 1977 .

[16]  John W Philbeck,et al.  Knowledge about typical source output influences perceived auditory distance. , 2002, The Journal of the Acoustical Society of America.

[17]  Rozenn Nicol,et al.  Perceptual Evaluation of Binaural Sound Synthesis: the Problem of Reporting Localization Judgments , 2003 .

[18]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[19]  Peter Rutherford,et al.  THE APPLICATION OF VIRTUAL ACOUSTIC TECHNIQUES FOR THE DEVELOPMENT OF AN AUDITORY NAVIGATION BEACON USED IN BUILDING EMERGENCY EGRESS , 2001 .

[20]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[21]  Chris Rizos,et al.  Syren - A Ship Based Location-Aware Audio Experience , 2005 .

[22]  Durand R. Begault,et al.  3-D Sound for Virtual Reality and Multimedia Cambridge , 1994 .

[23]  Constance Fleuriot,et al.  Children and emerging wireless technologies: investigating the potential for spatial practice , 2005, CHI.

[24]  Robert W. Lindeman,et al.  A classification scheme for multi-sensory augmented reality , 2007, VRST '07.

[25]  Tapio Lokki,et al.  Techniques and Applications of Wearable Augmented Reality Audio , 2003 .

[26]  Keith S. Jones,et al.  What Is an Affordance? , 2003, How Shall Affordances be Refined? Four Perspectives.

[27]  Simon Holland,et al.  AudioGPS: spatial audio in a minimal attention interface , 2001 .

[28]  Nikos Moustakas,et al.  Interactive Audio Realities: An Augmented / Mixed Reality Audio Game Prototype , 2011 .

[29]  David R. Morse,et al.  AudioGPS: Spatial Audio Navigation with a Minimal Attention Interface , 2002, Personal and Ubiquitous Computing.

[30]  Nick Mariette.  MITIGATION OF BINAURAL FRONT-BACK CONFUSIONS BY BODY MOTION IN AUDIO AUGMENTED REALITY , 2007 .

[31]  Roberta L. Klatzky,et al.  Personal guidance system for the visually impaired , 1994, ASSETS.

[32]  Mark R. Anderson,et al.  Direct comparison of the impact of head tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. , 2001, Journal of the Audio Engineering Society. Audio Engineering Society.

[33]  Chris Kyriakakis.  Fundamental and technological limitations of immersive audio systems , 1998 .

[34]  Tapio Lokki,et al.  A case study of auditory navigation in virtual acoustic environments , 2000 .

[35]  Miikka Tikander,et al.  Usability Issues in Listening to Natural Sounds with an Augmented Reality Audio Headset , 2009 .

[36]  Benjamin B. Bederson,et al.  Audio augmented reality: a prototype automated tour guide , 1995, CHI 95 Conference Companion.

[37]  F L Wightman,et al.  Resolution of front-back ambiguity in spatial hearing by listener and source movement. , 1999, The Journal of the Acoustical Society of America.

[38]  Joseph Michael Rozier.  Here&There : an augmented reality system of linked audio , 2000 .

[39]  L. A. Jeffress,et al.  Lateralization vs Localization , 1960 .

[40]  Geoffrey Holmes,et al.  Navigation-by-Music for Pedestrians: an Initial Prototype and Evaluation , 2006 .

[41]  Virginia Ann Best,et al.  Spatial Hearing with Simultaneous Sound Sources: A Psychophysical Investigation , 2004 .

[42]  Aki Härmä,et al.  Ambient Human-to-Human Communication , 2010, Handbook of Ambient Intelligence and Smart Environments.

[43]  Chris Schmandt,et al.  Design of spatialized audio in nomadic environments , 1997 .

[44]  Brian D. Simpson,et al.  Effects of Temporal Fine Structure on the Localization of Broadband Sounds: Potential Implications for the Design of Spatial Audio Displays , 2008 .

[45]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[46]  Malik Mallem.  Augmented Reality: Issues, trends and challenges , 2010, 2010 2nd International Conference on Image Processing Theory, Tools and Applications.

[47]  J M Loomis,et al.  Active localization of virtual sounds. , 1990, The Journal of the Acoustical Society of America.

[48]  Elizabeth M. Wenzel,et al.  Analysis of the Role of Update Rate and System Latency in Interactive Virtual Acoustic Environments , 1997 .

[49]  Jack M. Loomis,et al.  Auditory distance perception by translating observers , 1993, Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium.

[50]  Bruce N. Walker,et al.  Navigation performance in a virtual environment with bonephones , 2005 .

[51]  Simon Carlile,et al.  Contrasting monaural and interaural spectral cues for human sound localization. , 2004, The Journal of the Acoustical Society of America.