Enabling independent navigation for visually impaired people through a wearable vision-based feedback system

This work introduces a wearable system to provide situational awareness for blind and visually impaired people. The system includes a camera, an embedded computer and a haptic device to provide feedback when an obstacle is detected. The system uses techniques from computer vision and motion planning to (1) identify walkable space; (2) plan step-by-step a safe motion trajectory in the space, and (3) recognize and locate certain types of objects, for example the location of an empty chair. These descriptions are communicated to the person wearing the device through vibrations. We present results from user studies with low- and high-level tasks, including walking through a maze without collisions, locating a chair, and walking through a crowded environment while avoiding people.

[1]  João Barroso,et al.  Blind Navigation Support System based on Microsoft Kinect , 2012, DSAI.

[2]  Roberto Manduchi,et al.  (Computer) vision without sight , 2012, Commun. ACM.

[3]  Joel A. Hesch,et al.  Design and Analysis of a Portable Indoor Localization Aid for the Visually Impaired , 2010, Int. J. Robotics Res..

[4]  Nathan Ickes,et al.  A 0.6V, 8mW 3D Vision Processor for a Navigation Device for the Visually Impaired , 2016 .

[5]  Sven Koenig,et al.  Fast replanning for navigation in unknown terrain , 2005, IEEE Transactions on Robotics.

[6]  Zhengyou Zhang,et al.  Auditory augmented reality: Object sonification for the visually impaired , 2012, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP).

[7]  Gérard G. Medioni,et al.  Wearable RGBD Indoor Navigation System for the Blind , 2014, ECCV Workshops.

[8]  Yingli Tian,et al.  RGB-D Sensor-Based Computer Vision Assistive Technology for Visually Impaired Persons , 2014 .

[9]  Wolfram Burgard,et al.  Navigating blind people with a smart walker , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Stefan Roth,et al.  Stixmantics: A Medium-Level Model for Real-Time Semantic Scene Understanding , 2014, ECCV.

[11]  Nikolaos G. Bourbakis,et al.  Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Ming-Yu Liu,et al.  Layered Interpretation of Street View Images , 2015, Robotics: Science and Systems.

[13]  Ramiro Velazquez,et al.  Wearable Assistive Devices for the Blind , 2016, ArXiv.

[14]  Edwin Olson,et al.  LCM: Lightweight Communications and Marshalling , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  James M. Coughlan,et al.  Manhattan World: Orientation and Outlier Detection by Bayesian Inference , 2003, Neural Computation.

[16]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[17]  Uwe Franke,et al.  Pixels, Stixels, and Objects , 2012, ECCV Workshops.

[18]  James R. Marston,et al.  Attitudes of Visually Impaired Persons toward the Use of Public Transportation , 1997 .

[19]  Jianxiong Xiao,et al.  Sliding Shapes for 3D Object Detection in Depth Images , 2014, ECCV.

[20]  Henrik I. Christensen,et al.  Guidance for human navigation using a vibro-tactile belt interface and robot-like motion planning , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[21]  G. Medioni,et al.  RGB-D camera Based Navigation for the Visually Impaired , 2011 .

[22]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[23]  Roberto Manduchi,et al.  Mobile Vision as Assistive Technology for the Blind: An Experimental Study , 2012, ICCHP.

[24]  Uwe Franke,et al.  The Stixel World - A Compact Medium Level Representation of the 3D-World , 2009, DAGM-Symposium.

[25]  Adrian F. Clark,et al.  A Navigation System for the Visually Impaired: A Fusion of Vision and Depth Sensor , 2015, Applied bionics and biomechanics.

[26]  Sebastian Thrun,et al.  Learning user models of mobility-related activities through instrumented walking aids , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.