Usability and Effectiveness of Auditory Sensory Substitution Models for the Visually Impaired

This paper focuses on auditory sensory substitution for providing visually impaired users with suitable information in both static scene recognition and dynamic obstacle avoidance. We introduce three different sonification models together with three temporal presentation schemes, i.e. ways of temporally organizing the sonic events in order to provide suitable information. Following an overview of the motivation and challenges behind each of the solutions, we describe their implementation and an evaluation of their relative strengths and weaknesses based on a set of experiments conducted in a virtual environment.

[1]  Davide Rocchesso,et al.  A toolkit for explorations in sonic interaction design , 2010, Audio Mostly Conference.

[2]  Davide Rocchesso,et al.  Controlling Material Properties in Physical Models of Sounding Objects , 2001, ICMC.

[3]  I. Xenakis,et al.  Formalized Music: Thought and Mathematics in Composition , 1971 .

[4]  Mark R. Anderson,et al.  Direct comparison of the impact of head tracking, reverberation, and individualized head-related transfer functions on the spatial perception of a virtual speech source. , 2001, Journal of the Audio Engineering Society. Audio Engineering Society.

[5]  Kees van den Doel,et al.  Physically based models for liquid sounds , 2005, TAP.

[6]  Simone Spagnol,et al.  MODEL-BASED OBSTACLE SONIFICATION FOR THE NAVIGATION OF VISUALLY IMPAIRED PERSONS , 2016 .

[7]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[8]  William W. Gaver,et al.  Chapter 42 – Auditory Interfaces , 1997 .

[9]  William W. Gaver.  Auditory Icons: Using Sound in Computer Interfaces , 1986, Hum. Comput. Interact..

[10]  M. Minnaert.  XVI.On musical air-bubbles and the sounds of running water , 1933 .

[11]  Pawel Strumillo,et al.  Measurement System for Personalized Head-Related Transfer Functions and Its Verification by Virtual Source Localization Trials with Visually Impaired and Sighted Individuals , 2010 .

[12]  Curtis Roads,et al.  Introduction to Granular Synthesis , 1988 .

[13]  Stefan Bilbao.  Numerical Sound Synthesis: Finite Difference Schemes and Simulation in Musical Acoustics , 2009 .

[14]  Simone Spagnol,et al.  Structural modeling of pinna-related transfer functions , 2010 .

[15]  Simone Spagnol,et al.  Mixed structural modeling of head-related transfer functions for customized binaural audio delivery , 2013, 2013 18th International Conference on Digital Signal Processing (DSP).

[16]  György Wersényi,et al.  Sound of Vision - Spatial Audio Output and Sonification Approaches , 2016, ICCHP.

[17]  W. G. Gardner,et al.  HRTF measurements of a KEMAR , 1995 .

[18]  Mta Sztaki,et al.  Overview of Auditory Representations in Human-Machine Interfaces , 2013 .