Efficient Disparity Map Generation Using Stereo and Time-of-Flight Depth Cameras

Three-dimensional content 3D creation has received a lot of attention due to numerous successes of 3D entertainment. Accurate estimation of depth information is necessary for efficient 3D content creation. In this paper, we propose a disparity map estimation method based on stereo correspondence. The proposed system utilizes depth and stereo camera sets. While the stereo set carries out disparity estimation, depth camera information is projected to left and right camera positions using 3D warping and upsampling is processed in accordance with the image size. The upsampled depth is used for obtaining disparity data of left and right positions. Finally, disparity data from each depth sensor are combined. The experimental results demonstrate that our method produces more accurate disparity maps compared to the conventional approaches which use stereo cameras and a single depth sensor.

[1]  Liang Zhang,et al.  Stereoscopic image generation based on depth images for 3D TV , 2005, IEEE Transactions on Broadcasting.

[2]  Narendra Ahuja,et al.  A constant-space belief propagation algorithm for stereo matching , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Yo-Sung Ho,et al.  Discontinuity preserving disparity estimation with occlusion handling , 2014, J. Vis. Commun. Image Represent..

[5]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[6]  Qingxiong Yang,et al.  Near Real-time Stereo for Weakly-Textured Scenes , 2008, BMVC.

[7]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Jonathan M. Garibaldi,et al.  Real-Time Correlation-Based Stereo Vision with Reduced Border Errors , 2002, International Journal of Computer Vision.

[9]  Yo-Sung Ho,et al.  Generation of multi-view video using a fusion camera system for 3D displays , 2010, IEEE Transactions on Consumer Electronics.

[10]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[11]  Yo-Sung Ho,et al.  Generation of Multiview Images Using Stereo and Time-of-Flight Depth Cameras , 2013 .

[12]  Thomas Wiegand,et al.  Evaluation of view synthesis algorithms for mobile 3DTV , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[13]  Olga Veksler,et al.  Fast variable window for stereo correspondence using integral images , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Yo-Sung Ho,et al.  Efficient disparity map estimation using occlusion handling for various 3D multimedia applications , 2011, IEEE Transactions on Consumer Electronics.

[15]  Yo-Sung Ho,et al.  Generation of high-quality depth maps using hybrid camera system for 3-D video , 2011, J. Vis. Commun. Image Represent..

[16]  Yo-Sung Ho,et al.  High-quality multi-view depth generation using multiple color and depth cameras , 2010, 2010 IEEE International Conference on Multimedia and Expo.