Content-Aware Video Analysis to Guide Visually Impaired Walking on the Street

Although many researchers have developed systems or tools to assist blind and visually impaired people, they continue to face many obstacles in daily life—especially in outdoor environments. When people with visual impairments walk outdoors, they must be informed of objects in their surroundings. However, it is challenging to develop a system that can handle related tasks. In recent years, deep learning has enabled the development of many architectures with more accurate results than machine learning. One popular model for instance segmentation is Mask-RCNN, which can do segmentation and rapidly recognize objects. We use Mask-RCNN to develop a context-aware video that can help blind and visually impaired people recognize objects in their surroundings. Moreover, we provide the distance between the subject and object, and the object’s relative speed and direction using Mask-RCNN outputs. The results of our content-aware video include the name of the object, class object score, the distance between the person and the object, speed of the object, and object direction.

[1]  Fitri Utaminingrum,et al.  Building Segmentation of Satellite Image based on Area and Perimeter using Region Growing , 2016 .

[2]  Atma Ram Gupta,et al.  Smart Stick for the Blind and Visually Impaired People , 2018, 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT).

[3]  Jianyu Yang,et al.  Local Fast R-CNN Flow for Object-Centric Event Recognition in Complex Traffic Scenes , 2017, PSIVT Workshops.

[4]  Daniel Rocha,et al.  MyEyes-automatic combination system of clothing parts to blind people: First insights , 2017, 2017 IEEE 5th International Conference on Serious Games and Applications for Health (SeGAH).

[5]  Jinqiang Bai,et al.  Virtual-Blind-Road Following-Based Wearable Navigation Device for Blind People , 2018, IEEE Transactions on Consumer Electronics.

[6]  Hongwen He,et al.  Deep Learning for Vehicle Speed Prediction , 2018, Energy Procedia.

[7]  Jenq-Neng Hwang,et al.  Single-Camera and Inter-Camera Vehicle Tracking and 3D Speed Estimation Based on Fusion of Visual and Semantic Features , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Arnesh Sen,et al.  Ultrasonic Blind Stick for Completely Blind People to Avoid Any Kind of Obstacles , 2018, 2018 IEEE SENSORS.

[9]  Trung-Hieu Le,et al.  Hand segmentation under different viewpoints by combination of Mask R-CNN with tracking , 2018, 2018 5th Asian Conference on Defense Technology (ACDT).

[10]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Huang-Chia Shih,et al.  A Survey of Content-Aware Video Analysis for Sports , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Luis Miguel Bergasa,et al.  Unifying Terrain Awareness for the Visually Impaired through Real-Time Semantic Segmentation , 2018, Sensors.

[13]  Jeff S. Shamma,et al.  Vehicle Classification and Speed Estimation Using Combined Passive Infrared/Ultrasonic Sensors , 2018, IEEE Transactions on Intelligent Transportation Systems.

[14]  Praveen M Dhulavvagol,et al.  Vehical Tracking and Speed Estimation of Moving Vehicles for Traffic Surveillance Applications , 2017, 2017 International Conference on Current Trends in Computer, Electrical, Electronics and Communication (CTCEEC).

[15]  Zbigniew Czapla,et al.  Vehicle speed estimation with the use of gradient-based image conversion into binary form , 2017, 2017 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA).

[16]  Juarez Monteiro,et al.  Virtual guide dog: An application to support visually-impaired people through deep convolutional neural networks , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[17]  Vincent Lepetit,et al.  On Pre-Trained Image Features and Synthetic Images for Deep Learning , 2017, ECCV Workshops.

[18]  Yugyung Lee,et al.  Utilizing Mask R-CNN for Detection and Segmentation of Oral Diseases , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[19]  Abdelilah Maach,et al.  Speed estimation using simple line , 2018 .

[20]  Vikky Mohane,et al.  Object recognition for blind people using portable camera , 2016, 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (Startup Conclave).

[21]  Rama Chellappa,et al.  A Semi-Automatic 2D Solution for Vehicle Speed Estimation from Monocular Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).