Computer Vision-based System for Impaired Human Vision Compensation

This paper presents a high-level architecture of a computer vision-based system for partial compensation of lost or impaired human vision. It combines standard smartphone device, external deep learning-based image processing infrastructure and audio/tactile user interface. The proposed architecture is based on input from user-centered design process, involving end-users into system development. The paper discusses user needs and expectations for electronic travelling aids for the blind and highlights limitations of the existing solutions. The suggested architecture may be used as a basis for developing computer vision-based tools for visually impaired individuals. Keywords–computer vision; deep learning; mobile application; aid for blind and visually impaired; audio feedback; tactile feedback; impaired human vision compensation; user-centered design.

[1]  Larisa Dunai,et al.  Obstacle detectors for visually impaired people , 2014, 2014 International Conference on Optimization of Electrical and Electronic Equipment (OPTIM).

[2]  Mary Beth Rosson,et al.  Participatory design in community informatics , 2007 .

[3]  Stefano Mattoccia,et al.  A wearable mobility aid for the visually impaired based on embedded 3D vision and deep learning , 2016, 2016 IEEE Symposium on Computers and Communication (ISCC).

[4]  Nassir Navab,et al.  Deeper Depth Prediction with Fully Convolutional Residual Networks , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[5]  Chieko Asakawa,et al.  Personalized Dynamics Models for Adaptive Assistive Navigation Interfaces , 2018, ArXiv.

[6]  Narayanan Vijaykrishnan,et al.  The Third Eye: A Shopping Assistant for the Visually Impaired , 2017, CHI Extended Abstracts.

[7]  Tony Stockman,et al.  A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research , 2015, Journal on Multimodal User Interfaces.

[8]  Chenxi Liu,et al.  Attention Correctness in Neural Image Captioning , 2016, AAAI.

[9]  Cordelia Schmid,et al.  Long-Term Temporal Convolutions for Action Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jesper Simonsen,et al.  MUST: A Method for Participatory Design , 1998, Hum. Comput. Interact..

[12]  Bradford B. Glade,et al.  The Horus System , 1993 .

[13]  Adrian Burlacu,et al.  Computer Vision for the Visually Impaired: the Sound of Vision System , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[14]  Ali Hayek,et al.  Smart Assistive Navigation System for Blind and Visually Impaired Individuals , 2015, 2015 International Conference on Advances in Biomedical Engineering (ICABME).

[15]  Mahadev Satyanarayanan,et al.  OpenFace: A general-purpose face recognition library with mobile applications , 2016 .

[16]  Gretchen A. Stevens,et al.  Magnitude, temporal trends, and projections of the global prevalence of blindness and distance and near vision impairment: a systematic review and meta-analysis. , 2017, The Lancet. Global health.

[17]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.