A depth optimization method for 2D-to-3D conversion based on RGB-D images

2D-to-3D conversion is one of the key technologies in stereoscopic video. Its visual qualities depend on the qualities of depth image estimation and virtual viewpoint image rendering. Although there are many methods to get depth information, the depth information still has some errors because of calculation errors and noises, which influences the visual qualities of the generated virtual viewpoint image. This paper proposes a depth image optimization method to improve the qualities of virtual viewpoint image. It utilizes the surface normal information to segment objects and obtain the smooth factor of depth image. The experimental results demonstrate that our method can denoise and smooth the original depth map effectively.

[1]  Liang-Gee Chen,et al.  Depth Map Generation for 2D-to-3D Conversion by Short-Term Motion Assisted Color Segmentation , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[2]  Kourosh Khoshelham,et al.  Accuracy analysis of kinect depth data , 2012 .

[3]  Hideo Saito,et al.  Joint upsampling and noise reduction for real-time depth map enhancement , 2014, Electronic Imaging.

[4]  Hu Tian,et al.  Recovering depth of background and foreground from a monocular video with camera motion , 2013, 2013 Visual Communications and Image Processing (VCIP).

[5]  Aljoscha Smolic,et al.  View Synthesis for Advanced 3D Video Systems , 2008, EURASIP J. Image Video Process..

[6]  Kwanghoon Sohn,et al.  A Stereoscopic Video Generation Method Using Stereoscopic Display Characterization and Motion Analysis , 2008, IEEE Transactions on Broadcasting.

[7]  Oscar C. Au,et al.  Depth map denoising using graph-based transform and group sparsity , 2013, 2013 IEEE 15th International Workshop on Multimedia Signal Processing (MMSP).

[8]  Luis Salgado,et al.  Efficient spatio-temporal hole filling strategy for Kinect depth maps , 2012, Electronic Imaging.

[9]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Liang-Gee Chen,et al.  A 2D-to-3D conversion system using edge information , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[11]  Thomas Wiegand,et al.  3-D Video Representation Using Depth Maps , 2011, Proceedings of the IEEE.

[12]  Jitendra Malik,et al.  Object Detection in RGB-D Indoor Scenes 1 , 2013 .

[13]  Carlos Vázquez,et al.  CRC-CSDM: 2D-to-3D conversion using colour-based surrogate depth maps , 2010 .

[14]  Yo-Sung Ho,et al.  Depth image filter for mixed and noisy pixel removal in RGB-D camera systems , 2013, IEEE Transactions on Consumer Electronics.

[15]  Wen Gao,et al.  2D to 3D convertion based on edge defocus and segmentation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Liang-Gee Chen,et al.  A block-based 2D-to-3D conversion system with bilateral filter , 2009, 2009 Digest of Technical Papers International Conference on Consumer Electronics.

[17]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[18]  Liang-Gee Chen,et al.  A novel 2Dd-to-3D conversion system using edge information , 2010, IEEE Transactions on Consumer Electronics.

[19]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[20]  Wenjun Zeng,et al.  Kinect-like depth denoising , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[21]  Dongxiao Li,et al.  A Depth Extraction Method Based on Motion and Geometry for 2D to 3D Conversion , 2009, 2009 Third International Symposium on Intelligent Information Technology Application.

[22]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[23]  Wei Wu,et al.  Depth Template Based 2D-to-3D Video Conversion and Coding System , 2012, 2012 IEEE International Conference on Multimedia and Expo.