Wearable RGBD Indoor Navigation System for the Blind

In this paper, we present a novel wearable RGBD camera based navigation system for the visually impaired. The system is composed of a smartphone user interface, a glass-mounted RGBD camera device, a real-time navigation algorithm, and haptic feedback system. A smartphone interface provides an effective way to communicate to the system using audio and haptic feedback. In order to extract orientational information of the blind users, the navigation algorithm performs real-time 6-DOF feature based visual odometry using a glass-mounted RGBD camera as an input device. The navigation algorithm also builds a 3D voxel map of the environment and analyzes 3D traversability. A path planner of the navigation algorithm integrates information from the egomotion estimation and mapping and generates a safe and an efficient path to a waypoint delivered to the haptic feedback system. The haptic feedback system consisting of four micro-vibration motors is designed to guide the visually impaired user along the computed path and to minimize cognitive loads. The proposed system achieves real-time performance at \(28.4\)Hz in average on a laptop, and helps the visually impaired extends the range of their activities and improve the mobility performance in a cluttered environment. The experiment results show that navigation in indoor environments with the proposed system avoids collisions successfully and improves mobility performance of the user compared to conventional and state-of-the-art mobility aid devices.

[1]  Gérard G. Medioni,et al.  Robot vision for the visually impaired , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[2]  Gary S. Rubin,et al.  How does visual impairment affect performance on tasks of everyday life? SEE project. , 2001 .

[3]  Karen Bandeen-Roche,et al.  How does visual impairment affect performance on tasks of everyday life? The SEE Project , 2002 .

[4]  Roberto Manduchi,et al.  Dynamic environment exploration using a virtual white cane , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Sven Koenig,et al.  Fast replanning for navigation in unknown terrain , 2005, IEEE Transactions on Robotics.

[6]  Nassir Navab,et al.  Adaptive neighborhood selection for real-time surface normal estimation from organized point cloud data using integral images , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Per Ola Kristensson,et al.  Supporting blind navigation using depth sensing and sonification , 2013, UbiComp.

[8]  Iwan Ulrich,et al.  The GuideCane-a computerized travel aid for the active guidance of blind pedestrians , 1997, Proceedings of International Conference on Robotics and Automation.

[9]  Hans P. Moravec,et al.  High resolution maps from wide angle sonar , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[10]  Harald Reiterer,et al.  NAVI - A Proof-of-Concept of a Mobile Navigational Aid for Visually Impaired Based on the Microsoft Kinect , 2011, INTERACT.

[11]  Albert S. Huang,et al.  Visual Odometry and Mapping for Autonomous Flight Using an RGB-D Camera , 2011, ISRR.

[12]  Jonathan M. Garibaldi,et al.  Fast, unconstrained camera motion estimation from stereo without tracking and robust statistics , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[13]  John J. Leonard,et al.  Cooperative AUV Navigation Using a Single Surface Craft , 2009, FSR.

[14]  M. Lethbridge-Çejku,et al.  Summary health statistics for U.S. adults: National Health Interview Survey, 2006. , 2007, Vital and health statistics. Series 10, Data from the National Health Survey.

[15]  Wolfram Burgard,et al.  OctoMap : A Probabilistic , Flexible , and Compact 3 D Map Representation for Robotic Systems , 2010 .

[16]  Ian D. Reid,et al.  A Constant-Time Efficient Stereo SLAM System , 2009, BMVC.

[17]  Jack M. Loomis,et al.  GPS-Based Navigation Systems for the Visually Impaired , 2001 .

[18]  C I Howarth,et al.  The efficiency and walking speed of visually impaired people. , 1986, Ergonomics.

[19]  Shahram Izadi,et al.  Modeling Kinect Sensor Noise for Improved 3D Reconstruction and Tracking , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[20]  B. Munoz,et al.  Association of Visual Field Loss and Mobility Performance in Older Adults: Salisbury Eye Evaluation Study , 2004, Optometry and vision science : official publication of the American Academy of Optometry.

[21]  R Legood,et al.  Are we blind to injuries in the visually impaired? A review of the literature , 2002, Injury prevention : journal of the International Society for Child and Adolescent Injury Prevention.

[22]  Rosen Ivanov,et al.  Indoor navigation system for visually impaired , 2010, CompSysTech '10.

[23]  James R. Marston,et al.  Attitudes of Visually Impaired Persons toward the Use of Public Transportation , 1997 .

[24]  Andrew Howard,et al.  Real-time stereo visual odometry for autonomous ground vehicles , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Abdelsalam Helal,et al.  Drishti: an integrated navigation system for visually impaired and disabled , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[26]  Juan Manuel Sáez,et al.  6DOF entropy minimization SLAM for stereo-based wearable devices , 2011, Comput. Vis. Image Underst..

[27]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[28]  Christian Laugier,et al.  Update Policy of Dense Maps: Efficient Algorithms and Sparse Representation , 2007, FSR.

[29]  Juan Manuel Saez Martinez,et al.  Stereo-based Aerial Obstacle Detection for the Visually Impaired , 2008 .

[30]  G. Medioni,et al.  RGB-D camera Based Navigation for the Visually Impaired , 2011 .