Real Time Path Finding for Assisted Living Using Deep Learning

The paper presents a computer vision based system, which performs real time path finding for visually impaired or blind people. The semantic segmentation of camera images is performed using deep convolutional neural network (CNN), which able to recognize patterns across image feature space. Out of three different CNN architectures (AlexNet, GoogLeNet and VGG) analysed, the fully connected VGG16 neural network is shown to perform best in the semantic segmentation task. The algorithm for extracting and finding paths, obstacles and path boundaries is presented. The experiments performed using own dataset (300 images extracted from two hours of video recording walking in outdoors environment) show that the developed system is able to find paths, path objects and path boundaries with an accuracy of 96.1 ± 2.6%.

[1]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Andrei Bursuc,et al.  A Smartphone-Based Obstacle Detection and Classification System for Assisting Visually Impaired People , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[3]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Nikolaos G. Bourbakis,et al.  Towards a 2D tactile vocabulary for navigation of blind and visually impaired , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[5]  Torsten Wörtwein,et al.  Mobile Interactive Image Sonification for the Blind , 2016, ICCHP.

[6]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Marcin Gabryel,et al.  The Image Classification with Different Types of Image Features , 2017, ICAISC.

[8]  Bing Li,et al.  Assisting blind people to avoid obstacles: An wearable obstacle stereo feedback system based on 3D detection , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[9]  Kwangsu Cho,et al.  Usability and Design Guidelines of Smart Canes for Users with Visual Impairments , 2013 .

[10]  Adam Gordois,et al.  The global economic cost of visual impairment , 2010 .

[11]  Byeong-Seok Shin,et al.  Obstacle Detection and Avoidance System for Visually Impaired People , 2007, HAID.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Luis Miguel Bergasa,et al.  Assisting the Visually Impaired: Obstacle Detection and Warning System by Acoustic Feedback , 2012, Sensors.

[14]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[15]  Juan Manuel Saez Martinez,et al.  Stereo-based Aerial Obstacle Detection for the Visually Impaired , 2008 .

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Juan Manuel Sáez,et al.  First Steps towards Stereo-based 6DOF SLAM for the Visually Impaired , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[18]  Jack M. Loomis,et al.  GPS-Based Navigation Systems for the Visually Impaired , 2001 .

[19]  Yassine Salih Alj,et al.  Obstacles Recognition System for the Blind People Using RFID , 2012, 2012 Sixth International Conference on Next Generation Mobile Applications, Services and Technologies.

[20]  Roy Paily,et al.  Blind Navigation Assistance for Visually Impaired based on Local Depth Hypothesis from a Single Image , 2013 .

[21]  N. A. Bradley,et al.  Assistive Technology For Visually Impaired And Blind People , 2008 .

[22]  Christian Napoli,et al.  Using AOP Neural Networks to Infer User Behaviours and Interests , 2015, WOA.

[23]  R. Manduchi,et al.  Mobility-Related Accidents Experienced by People with Visual Impairment , 2010 .

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Gérard G. Medioni,et al.  Robot vision for the visually impaired , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[26]  Robertas Damasevicius,et al.  Human Activity Recognition in AAL Environments Using Random Projections , 2016, Comput. Math. Methods Medicine.

[27]  Martin Lauer,et al.  A generative model for 3D urban scene understanding from movable platforms , 2011, CVPR 2011.