Registration and fusion for ToF camera and 2D camera reading

Time of Flight (ToF) cameras have become a competitive alternative to traditional distance sensing techniques like laser or stereo vision as it can deliver grayscale images and 3D information simultaneously at high frame-rates. However the low resolution of ToF camera limits the applications for accurate segmentation or classification. This paper presents a fast and robust solution to combine the 3D information of ToF camera and high resolution color image. Firstly, we setup a 2D/3D stereo camera with fixed spatial relation and similar visual field. Based on characteristic of ToF camera and the principles of stereo vision system, the 3D information of ToF camera is registered with high resolution color image by matching the high resolution color image with grayscale image of ToF camera. This method has very low computation cost while the matching accuracy is only determined by physical characteristics or parameters of the 2D/3D stereo camera system without extra computation error. Experiments results have demonstrated the feasibility, efficiency and accuracy of proposed algorithms.

[1]  Andreas Birk,et al.  Efficient Representation in Three-Dimensional Environment Modeling for Planetary Robotic Exploration , 2010, Adv. Robotics.

[3]  Rasmus Larsen,et al.  Fusion of stereo vision and Time-Of-Flight imaging for improved 3D estimation , 2008, Int. J. Intell. Syst. Technol. Appl..

[4]  Zhengyou Zhang,et al.  Low-complexity, near-lossless coding of depth maps from kinect-like depth cameras , 2011, 2011 IEEE 13th International Workshop on Multimedia Signal Processing.

[5]  Paul Timothy Furgale,et al.  Field testing of a rover guidance, navigation, and control architecture to support a ground-ice prospecting mission to Mars , 2011, Robotics Auton. Syst..

[6]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[7]  Piotr Jasiobedzki,et al.  Lidar and the mobile Scene Modeler (mSM) as scientific tools for planetary exploration , 2010 .

[8]  Klaus-Dieter Kuhnert,et al.  Fusion of Stereo-Camera and PMD-Camera Data for Real-Time Suited Precise 3D Environment Reconstruction , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  U. Scheunert,et al.  Multi sensor fusion for object detection using generalized feature models , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[10]  Tobias Hanning,et al.  Calibration and low-level data fusion algorithms for a parallel 2D/3D-camera , 2011, Inf. Fusion.

[11]  Andreas Birk,et al.  Efficient Representation in 3D Environment Modeling for Planetary Robotic Exploration , 2010 .

[12]  S. E. Ghobadi,et al.  First steps in enhancing 3D vision technique using 2D/3D sensors , 2006 .

[13]  B. Huhle,et al.  Integrating 3D Time-of-Flight Camera Data and High Resolution Images for 3DTV Applications , 2007, 2007 3DTV Conference.

[14]  Bart C. Nabbe,et al.  Progress in 3-D Mapping and Localization , 2001 .

[15]  Reinhard Klette,et al.  Panoramic Mapping using CCD-Line Camera and Laser Scanner with Integrated Position and Orientation System , 2003 .