TOF-CCD image fusion using complex wavelets

A new generation of ToF cameras are practical devices capable of real-time 3D scene reconstruction. They are mainly limited by two factors; accuracy and low spatial resolution. Standard CCD and ToF images of a scene share likenesses in some regions, e.g., object edges are apparent as gradients in both modals. Here we propose an approach to overcome the resolution limitation; using an effective dual-tree complex wavelet transform framework in a calibrated setup to fuse the low resolution TOF image with the high resolution details of the CCD image. We show how this can enhance features such as at object borders.

[1]  N. Kingsbury.  Complex Wavelets for Shift Invariant Analysis and Filtering of Signals , 2001 .

[2]  Reinhard Koch,et al.  Integration of a Time-of-Flight Camera into a Mixed Reality System for Handling Dynamic Scenes, Moving Viewpoints and Occlusions in Real-Time , 2008 .

[3]  Reinhard Koch,et al.  ToF-sensors: New dimensions for realism and interactivity , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[4]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[5]  Stavri G. Nikolov,et al.  Image fusion: Advances in the state of the art , 2007, Inf. Fusion.

[6]  Richard Baraniuk,et al.  The dual-tree complex wavelet transform , 2005, IEEE Signal Processing Magazine.

[7]  Ruigang Yang,et al.  Fusion of time-of-flight depth and stereo for high accuracy depth maps , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Wolfgang Straßer,et al.  On-the-fly scene acquisition with a handy multi-sensor system , 2008, Int. J. Intell. Syst. Technol. Appl..

[9]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[10]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[11]  Rasmus Larsen,et al.  Fusion of stereo vision and Time-Of-Flight imaging for improved 3D estimation , 2008, Int. J. Intell. Syst. Technol. Appl..

[12]  R. Koch,et al.  CALIBRATION OF A PMD-CAMERA USING A PLANAR CALIBRATION PATTERN TOGETHER WITH A MULTI-CAMERA SETUP , 2008 .

[13]  Richard Baraniuk,et al.  The Dual-tree Complex Wavelet Transform , 2007 .