Real-time Obstacle Detection in Outdoor Environment for Visually Impaired using RGB-D and Disparity Map

RGB-D sensor is efficient for real-time obstacle detection, but it still does not work with direct sunlight. In this paper, we present a novel approach for real-time obstacle detection in an outdoor urban environment using stereo images and IR depth information based on dual Microsoft Kinect Xbox 360. Our system performed fast disparity mapping from each pair of images by the sum of absolute differences (SAD), which is a block-matching algorithm, then generated a robust 3D point cloud with the disparity map and IR depth information. Extraction the obstacle from the background was done using random-sample consensus (RANSAC) method. The experiments based on MATLAB R2016a involved comprehensive comparison with several alternative validation parameters including disparity block size, disparity range and morphological close element. The results show the algorithm runtime and error occluded pixel rate.

[1]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Atiwong Suchato,et al.  Directional Obstacle Warning Device Using Multiple Ultrasonic Transducers for People with Visual Disabilities , 2015 .

[3]  Nicolas Vuillerme,et al.  Real-Time Obstacle Detection System in Indoor Environment for the Visually Impaired Using Microsoft Kinect Sensor , 2016, J. Sensors.

[4]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[5]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[6]  Surapol Vorapatratorn,et al.  iSonar: an obstacle warning device for the totally blind , 2014 .

[7]  Luis Miguel Bergasa,et al.  Assisting the Visually Impaired: Obstacle Detection and Warning System by Acoustic Feedback , 2012, Sensors.

[8]  김만두,et al.  시력 손상과 시각 장애(Visual Impairment and Blindness) , 2011 .

[9]  Yingli Tian,et al.  RGB-D Sensor-Based Computer Vision Assistive Technology for Visually Impaired Persons , 2014 .

[10]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[11]  Gérard G. Medioni,et al.  Visual Navigation Aid for the Blind in Dynamic Environments , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[12]  Frank Dellaert,et al.  Map-based priors for localization , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[13]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[14]  Ching-Tang Hsieh,et al.  An Indoor Obstacle Detection System Using Depth Information and Region Growth , 2015, Sensors.

[15]  Josechu J. Guerrero,et al.  Detection and Modelling of Staircases Using a Wearable Depth Sensor , 2014, ECCV Workshops.

[16]  Ian P. Howard,et al.  Binocular Vision and Stereopsis , 1996 .