Improveing depth estimation through fusion of stereo and TOF

In this paper, a novel method for depth estimation through fusion of stereo and TOF is proposed. Both the depth data and intensity gray images from TOF are taken into the calibration processing for getting the more accurate camera exterior parameters. Using the exterior parameters, the depth image captured by TOF is warped to the scope of the CCD camera pairs to form an initial depth image. Due to the problems of depth errors in the edges and the occlusion, the stereo rig is used to obtain the accurate depth of the edges of the objects, and eliminates the effect of occlusion simultaneously. Finally, by fusion method that takes advantage of the stereo vision, the depth data and the color image, the depth is optimized to obtain high resolution depth image. The experiment results show that higher spatial resolution and correct depth at edges' position is obtained, compared with only using the TOF camera, and also the higher quality dense depth image.

[1]  Hilmar Ingensand,et al.  Calibration of the fast range imaging camera SwissRanger for use in the surveillance of the environment , 2006, SPIE Security + Defence.

[2]  Emanuele Trucco,et al.  Rectification with unconstrained stereo geometry , 1997, BMVC.

[3]  Ruigang Yang,et al.  Fusion of time-of-flight depth and stereo for high accuracy depth maps , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  S. Burak Gokturk,et al.  A Time-Of-Flight Depth Sensor - System Description, Issues and Solutions , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[5]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Timo Kahlmann,et al.  CALIBRATION FOR INCREASED ACCURACY OF THE RANGE IMAGING CAMERA SWISSRANGER , 2006 .

[7]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[8]  Tian Lei,et al.  An improved Otsu image segmentation algorithm for path mark detection under variable illumination , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[9]  Rasmus Larsen,et al.  Fusion of stereo vision and Time-Of-Flight imaging for improved 3D estimation , 2008, Int. J. Intell. Syst. Technol. Appl..

[10]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[11]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[12]  Sebastian Thrun,et al.  An Application of Markov Random Fields to Range Sensing , 2005, NIPS.

[13]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[14]  Stefan May,et al.  3D time-of-flight cameras for mobile robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..