3D video coding with depth modeling modes and view synthesis optimization

This paper presents efficient coding tools for depth data in depth-enhanced video formats. The method is based on the high-efficiency video codec (HEVC). The developed tools include new depth modeling modes (DMMs), in particular using non-rectangular wedgelet and contour block partitions. As the depth data is used for synthesis of new video views, a specific 3D video encoder optimization is used. This view synthesis optimization (VSO) considers the exact local distortion in a synthesized intermediate video portion or image block for the depth map coding. In a fully optimized 3D-HEVC coder, VSO achieves average bit rate savings of 17%, while DMMs gain 6%in BD rate, even though the depth rate only contributes 10% to the overall MVD bit rate.

[1]  Detlev Marpe,et al.  3D video: Depth coding based on inter-component prediction of block partitions , 2012, 2012 Picture Coding Symposium.

[2]  Jaejoon Lee,et al.  Depth Map Coding Based on Synthesized View Distortion Function , 2011, IEEE Journal of Selected Topics in Signal Processing.

[3]  Peter H. N. de With,et al.  Platelet-based coding of depth maps for the transmission of multiview images , 2006, Electronic Imaging.

[4]  Aljoscha Smolic,et al.  Efficient Prediction Structures for Multiview Video Coding , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Yo-Sung Ho,et al.  Depth Reconstruction Filter and Down/Up Sampling for Depth Coding in 3-D Video , 2009, IEEE Signal Processing Letters.

[6]  Ismo Rakkolainen,et al.  A Survey of 3DTV Displays: Techniques and Technologies , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Antonio Ortega,et al.  Depth map coding with distortion estimation of rendered view , 2010, Electronic Imaging.

[8]  Itu-T and Iso Iec Jtc.  Advanced video coding for generic audiovisual services , 2010 .

[9]  Edson M. Hung,et al.  Efficiency improvements for a geometric-partition-based video coder , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[10]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[11]  Heiko Schwarz,et al.  Motion vector inheritance for high efficiency 3D video plus depth coding , 2012, 2012 Picture Coding Symposium.

[12]  Heiko Schwarz,et al.  Inter-view prediction of motion data in multiview video coding , 2012, 2012 Picture Coding Symposium.

[13]  Gary J. Sullivan,et al.  Recent developments in standardization of high efficiency video coding (HEVC) , 2010, Optical Engineering + Applications.

[14]  Ying Chen,et al.  The Emerging MVC Standard for 3D Video Services , 2008, EURASIP J. Adv. Signal Process..

[15]  J. Konrad,et al.  3D displays and signal processing : an answer to 3-D ills? , 2007 .

[16]  Thomas Wiegand,et al.  3-D Video Representation Using Depth Maps , 2011, Proceedings of the IEEE.

[17]  Yo-Sung Ho,et al.  H.264-Based Depth Map Sequence Coding Using Motion Information of Corresponding Texture Video , 2006, PSIVT.

[18]  Sehoon Yea,et al.  View synthesis prediction for multiview video coding , 2009, Signal Process. Image Commun..

[19]  Aljoscha Smolic,et al.  The effects of multiview depth video compression on multiview rendering , 2009, Signal Process. Image Commun..

[20]  Heiko Schwarz,et al.  3D video coding using the synthesized view distortion change , 2012, 2012 Picture Coding Symposium.