Improving RealSense by Fusing Color Stereo Vision and Infrared Stereo Vision for the Visually Impaired

The introduction of RGB-D sensor has attracted attention from researchers majored in computer vision. With real-time depth measurement provided by RGB-D sensor, a better navigational assistance than traditional aiding tools can be offered for visually impaired people. However, nowadays RGB-D sensor usually has a limited detecting range, and fails in performing depth measurement on objects with special surfaces, such as absorbing, specular, and transparent surfaces. In this paper, a novel algorithm using two RealSense R200 simultaneously to build a short-baseline color stereo vision system is developed. This algorithm enhances depth estimation by fusing color stereo depth map and original RealSense depth map, which is obtained by infrared stereo vision. Moreover, the minimum range is decreased by up to 84.6%, from 650mm to 100mm. We anticipate out algorithm to provide better assistance for visually impaired individuals.

[1]  Edward Grant,et al.  The Kinect up close: Adaptations for short-range imaging , 2012, 2012 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[2]  Minh N. Do,et al.  Fast Global Image Smoothing Based on Weighted Least Squares , 2014, IEEE Transactions on Image Processing.

[3]  Emile A. Hendriks,et al.  Hybrid Kinect Depth Map Refinement for Transparent Objects , 2014, 2014 22nd International Conference on Pattern Recognition.

[4]  Marc Pollefeys,et al.  Real-Time Direct Dense Matching on Fisheye Images Using Plane-Sweeping Stereo , 2014, 2014 2nd International Conference on 3D Vision.

[5]  Jian Bai,et al.  Expanding the Detection of Traversable Area with RealSense for the Visually Impaired , 2016, Sensors.

[6]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  S. Yamaguchi,et al.  Kinect cane: An assistive system for the visually impaired based on three-dimensional object recognition , 2012, 2012 IEEE/SICE International Symposium on System Integration (SII).

[8]  Susan M. Downes,et al.  A Depth-Based Head-Mounted Visual Display to Aid Navigation in Partially Sighted Individuals , 2013, PloS one.

[9]  Mario Fritz,et al.  Improving the Kinect by Cross-Modal Stereo , 2011, BMVC.

[10]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  João Barroso,et al.  Assisted Guidance for the Blind Using the Kinect Device , 2016, DSAI.

[12]  Dong Liu,et al.  IR stereo RealSense: Decreasing minimum range of navigational assistance for visually impaired individuals , 2017, J. Ambient Intell. Smart Environ..

[13]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Harald Reiterer,et al.  NAVI - A Proof-of-Concept of a Mobile Navigational Aid for Visually Impaired Based on the Microsoft Kinect , 2011, INTERACT.

[15]  Mario Fritz,et al.  I spy with my little eye: Learning optimal filters for cross-modal stereo under projected patterns , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).