High quality depth maps from stereo matching and ToF camera

Depth generation is a key technology in computer vision. Known methods mainly rely on stereo matching, or measuring devices like ToF camera. The ToF camera performs well in low-textured regions and repetitive regions where stereo matching fails. In contrast, stereo matching works better than ToF camera in textured regions. Based on their complementary characteristics, we introduce a method to combine ToF depth and stereo matching. In order to integrate their respective advantages, we measure their reliabilities and construct a new cost volume. Experiment results show that our fusion algorithm improves the accuracy and robustness, the generated depth map is much better than that obtained from an individual method.

[1]  Ruigang Yang,et al.  Reliability Fusion of Time-of-Flight Depth and Stereo Geometry for High Quality Depth Maps , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Reinhard Koch,et al.  Time-of-Flight Sensors in Computer Graphics , 2009, Eurographics.

[3]  Reinhard Männer,et al.  Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation , 2004, International Journal of Computer Vision.

[4]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[5]  K. Hartmann,et al.  Data-Fusion of PMD-Based Distance-Information and High-Resolution RGB-Images , 2007, 2007 International Symposium on Signals, Circuits and Systems.

[6]  Reinhard Koch,et al.  A Comparison of PMD-Cameras and Stereo-Vision for the Task of Surface Reconstruction using Patchlets , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Ming Zhang,et al.  Fast stereo matching algorithm based on adaptive window , 2010, 2010 International Conference on Audio, Language and Image Processing.

[9]  John G. Apostolopoulos,et al.  Fusion of active and passive sensors for fast 3D capture , 2010, 2010 IEEE International Workshop on Multimedia Signal Processing.

[10]  H. Aanaes,et al.  Environmental Effects on Measurement Uncertainties of Time-of-Flight Cameras , 2007, 2007 International Symposium on Signals, Circuits and Systems.

[11]  Pascal Fua,et al.  A parallel stereo algorithm that produces dense depth maps and preserves image features , 1993, Machine Vision and Applications.

[12]  Rasmus Larsen,et al.  Fusion of stereo vision and Time-Of-Flight imaging for improved 3D estimation , 2008, Int. J. Intell. Syst. Technol. Appl..

[13]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Dongxiao Li,et al.  An Asymmetric Edge Adaptive Filter for Depth Generation and Hole Filling in 3DTV , 2010, IEEE Transactions on Broadcasting.

[15]  Gauthier Lafruit,et al.  Anisotropic local high-confidence voting for accurate stereo correspondence , 2008, Electronic Imaging.