An Environmental Perception and Navigational Assistance System for Visually Impaired Persons Based on Semantic Stixels and Sound Interaction

Assistive technologies aim at enhancing personal mobility of individuals with disabilities to improve their independence and access to social life. For the visually impaired, perception during navigation comprises a major ingredient of independent living. With the development of computer vision, it is possible to meet the richer needs of visually impaired people. However, research on navigation assistance for the visually impaired is still relatively unexplored when compared with the active progress in autonomous driving which is already in full swing. In respond to this issue, we aim to leverage the study of the Stixel-World for automotive systems and transfer it to develop assistive technology for visually impaired people. The impressive research results of deep learning also suppose benefits for vision-based technology. Precisely, semantic segmentation is a task that enables identification of different objects uniformly. Inspired by these observations, we design a set of wearable visual aids, while the core algorithm is based on the stixel representations for three-dimensional world combined with pixel-wise semantic segmentation. Predetermined conditions for stixels in automotive research, such as camera angles, position fixes, and unsuitable assumptions made about the real world are optimized to fit the needs of navigation assistance in our algorithm, along with the incorporation of traversability-related semantic information. We also propose a sound mapping scheme, so that the environmental awareness about geographic and semantic information are conveyed to the visually impaired through acoustic feedback.

[1]  Uwe Franke,et al.  The Stixel World - A Compact Medium Level Representation of the 3D-World , 2009, DAGM-Symposium.

[2]  Tugrul U. Daim,et al.  What will it take to adopt smart glasses: A consumer choice based review? , 2017 .

[3]  Luc Van Gool,et al.  Stixels Motion Estimation without Optical Flow Computation , 2012, ECCV.

[4]  Jennifer L. Milne,et al.  Shape-specific activation of occipital cortex in an early blind echolocation expert , 2013, Neuropsychologia.

[5]  Luis Miguel Bergasa,et al.  Unifying terrain awareness through real-time semantic segmentation , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[6]  Laura Giarré,et al.  Enabling independent navigation for visually impaired people through a wearable vision-based feedback system , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Eduardo Romera,et al.  ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation , 2018, IEEE Transactions on Intelligent Transportation Systems.

[8]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[9]  Jian Bai,et al.  Expanding the Detection of Traversable Area with RealSense for the Visually Impaired , 2016, Sensors.

[10]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, CVPR.

[11]  Shuicheng Yan,et al.  A survey on deep learning-based fine-grained object classification and semantic segmentation , 2017, International Journal of Automation and Computing.

[12]  T. Vaudrey,et al.  Stereo-based Free Space Computation in Complex Traffic Scenarios , 2008, 2008 IEEE Southwest Symposium on Image Analysis and Interpretation.

[13]  Rainer Stiefelhagen,et al.  Using Technology Developed for Autonomous Cars to Help Navigate Blind People , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[14]  Shuihua Wang,et al.  RGB-D image-based detection of stairs, pedestrian crosswalks and traffic signs , 2014, J. Vis. Commun. Image Represent..

[15]  N. Kopco,et al.  Neuronal representations of distance in human auditory cortex , 2012, Proceedings of the National Academy of Sciences.

[16]  Jian Bai,et al.  Detecting Traversable Area and Water Hazards for the Visually Impaired with a pRGB-D Sensor , 2017, Sensors.

[17]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Marc Pollefeys,et al.  Semantic Stixels: Depth is not enough , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[19]  Brian C. J. Moore,et al.  A summary of research investigating echolocation abilities of blind and sighted humans , 2014, Hearing Research.

[20]  Luis Miguel Bergasa,et al.  Unifying Terrain Awareness for the Visually Impaired through Real-Time Semantic Segmentation , 2018, Sensors.

[21]  Uwe Franke,et al.  Efficient representation of traffic scenes by means of dynamic stixels , 2010, 2010 IEEE Intelligent Vehicles Symposium.