Vision System with 3D Audio Feedback to assist Navigation for Visually Impaired

Abstract Freedom of movement, identification of humans and material objects, and interaction with the environment still pose a challenge in the development of assistive devices for the visually impaired. The design of assistive devices is generally accomplished by integration of sensors, vision systems, tactile, and audio feedback systems. This paper proposes a vision system with a 3D audio feedback mechanism, for reliable navigation by the visually impaired. The variations in sound intensity of earphones facilitate a natural and intuitive cognizance of the relative positions of objects, in a pathway. The proposed system is comprised of three main modules: depth calculation, where a depth map is found using stereoscopic vision; object detection, where a trained Convolutional Neural Network (CNN) is used to identify doors, and extract their location; and, 3D audio generation, which generates an audio vector, from the depth map and an object’s location. The audio vector is then used to generate a spatial audio signal for the user. This system proved to be useful and accurate for navigation by the visually impaired, when tested in a real-life environment.