Interactive Travel Aid for the Visually Impaired: from Depth Maps to Sonic Patterns and Verbal Messages

This paper presents user trials of a prototype micro-navigation aid for the visually impaired. The main advantage of the system is its small form factor. The device consists of a Structure Sensor depth camera, a smartphone, a remote controller and a pair of headphones. An original feature of the system is its interactivity. The user can activate different space scanning modes and different sound presentation schemes for 3D scenes on demand. The results of the trials are documented by timeline logs recording the activation of different interactive modes. The aim of the first trial was to test system capability for aiding the visually impaired to avoid obstacles. The second tested system efficiency at detecting open spaces. The two visually impaired testers performed the trials successfully, although the times required to complete the tasks seem rather long. Nevertheless, the trials show the potential usefulness of the system as a navigational aid and have enabled us to introduce numerous improvements to the tested prototype.

[1]  Junseok Kwon,et al.  Joint Tracking and Ground Plane Estimation , 2016, IEEE Signal Processing Letters.

[2]  Shachar Maidenbaum,et al.  Author's Personal Copy Neuroscience and Biobehavioral Reviews Sensory Substitution: Closing the Gap between Basic Research and Widespread Practical Visual Rehabilitation Author's Personal Copy , 2022 .

[3]  Anke M. Brock,et al.  Interactive audio-tactile maps for visually impaired people , 2015, ASAC.

[4]  Stephen A. Brewster,et al.  Constructing sonified haptic line graphs for the blind student: first steps , 2000, Assets '00.

[5]  Joseph Muscat.  Analytical Geometry , 1885, Nature.

[6]  Edwige E. Pissaloux,et al.  Mobility of visually impaired people : fundamentals and ICT assistive technologies , 2018 .

[7]  Pawel Strumillo,et al.  Interactive sonification of U-depth images in a navigation aid for the visually impaired , 2018, Journal on Multimodal User Interfaces.

[8]  M. Serrão,et al.  Computer vision and GIS for the navigation of blind persons in buildings , 2013, Universal Access in the Information Society.

[9]  Runar Unnthorsson,et al.  Different Approaches to Aiding Blind Persons in Mobility and Navigation in the “Naviton” and “Sound of Vision” Projects , 2018 .

[10]  K. Sathian,et al.  Multisensory cortical processing of object shape and its relation to mental imagery , 2004, Cognitive, affective & behavioral neuroscience.

[11]  Feng Guo,et al.  Road obstacle detection in stereo vision based on UV-disparity , 2014 .

[12]  N. A. Bradley,et al.  Assistive Technology For Visually Impaired And Blind People , 2008 .

[13]  Michal Bujacz,et al.  Sonification: Review of Auditory Display Solutions in Electronic Travel Aids for the Blind , 2016 .

[14]  Michael Banf,et al.  A Modular Computer Vision Sonification Model For The Visually Impaired , 2012 .

[15]  Charles O'Neill,et al.  Hearing Images: Interactive Sonification Interface for Images , 2008, 2008 International Conference on Automated Solutions for Cross Media Content and Multi-Channel Distribution.

[16]  Titus Zaharia,et al.  Wearable assistive devices for visually impaired: A state of the art survey , 2020, Pattern Recognit. Lett..

[17]  Constantine Stephanidis,et al.  Universal access in the information society , 1999, HCI.

[18]  Nikolaos G. Bourbakis,et al.  Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[19]  Pawel Strumillo,et al.  Measurement System for Personalized Head-Related Transfer Functions and Its Verification by Virtual Source Localization Trials with Visually Impaired and Sighted Individuals , 2010 .