Regional Bit Allocation and Rate Distortion Optimization for Multiview Depth Video Coding With View Synthesis Distortion Model

In this paper, we propose a view synthesis distortion model (VSDM) that establishes the relationship between depth distortion and view synthesis distortion for the regions with different characteristics: color texture area corresponding depth (CTAD) region and color smooth area corresponding depth (CSAD), respectively. With this VSDM, we propose regional bit allocation (RBA) and rate distortion optimization (RDO) algorithms for multiview depth video coding (MDVC) by allocating more bits on CTAD for rendering quality and fewer bits on CSAD for compression efficiency. Experimental results show that the proposed VSDM based RBA and RDO can improve the coding efficiency significantly for the test sequences. In addition, for the proposed overall MDVC algorithm that integrates VSDM based RBA and RDO, it achieves 9.99% and 14.51% bit rate reduction on average for the high and low bit rate, respectively. It can improve virtual view image quality 0.22 and 0.24 dB on average at the high and low bit rate, respectively, when compared with the original joint multiview video coding model. The RD performance comparisons using five different metrics also validate the effectiveness of the proposed overall algorithm. In addition, the proposed algorithms can be applied to both INTRA and INTER frames.

[1]  Antonio Ortega,et al.  Depth map distortion analysis for view rendering and depth coding , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[2]  Erhan Ekmekcioglu,et al.  Content Adaptive Enhancement of Multi-View Depth Maps for Free Viewpoint Video , 2011, IEEE Journal of Selected Topics in Signal Processing.

[3]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[4]  Dong Tian,et al.  New Depth Coding Techniques With Utilization of Corresponding Video , 2011, IEEE Transactions on Broadcasting.

[5]  Ahmet M. Kondoz,et al.  Utilisation of edge adaptive upsampling in compression of depth map videos for enhanced free-viewpoint rendering , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[6]  Minh N. Do,et al.  Depth Video Enhancement Based on Weighted Mode Filtering , 2012, IEEE Transactions on Image Processing.

[7]  Yo-Sung Ho,et al.  Depth Coding Using a Boundary Reconstruction Filter for 3-D Video Systems , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Dong Tian,et al.  Boundary Artifact Reduction in View Synthesis of 3D Video: From Perspective of Texture-Depth Alignment , 2011, IEEE Transactions on Broadcasting.

[9]  Antonio Ortega,et al.  Improving view rendering quality and coding efficiency by suppressing compression artifacts in depth-image coding , 2009, Electronic Imaging.

[10]  Warnakulasuriya Anil Chandana Fernando,et al.  Adaptive motion-estimation-mode selection for depth video coding , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Thomas Wiegand,et al.  Diffusion filtering of depth maps in stereo video coding , 2010, 28th Picture Coding Symposium.

[12]  Wen Gao,et al.  A Novel Rate Control Technique for Multiview Video Plus Depth Based 3D Video Coding , 2011, IEEE Transactions on Broadcasting.

[13]  Thomas Wiegand,et al.  3-D Video Representation Using Depth Maps , 2011, Proceedings of the IEEE.

[14]  Stephen P. Boyd,et al.  Joint optimization of communication rates and linear systems , 2003, IEEE Trans. Autom. Control..

[15]  Gary J. Sullivan,et al.  Overview of the Stereo and Multiview Video Coding Extensions of the H.264/MPEG-4 AVC Standard , 2011, Proceedings of the IEEE.

[16]  Zhenzhong Chen,et al.  Depth No-Synthesis-Error Model for View Synthesis in 3-D Video , 2011, IEEE Transactions on Image Processing.

[17]  Yo-Sung Ho,et al.  Multi-view depth video coding using depth view synthesis , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[18]  Jaejoon Lee,et al.  Depth Map Coding Based on Synthesized View Distortion Function , 2011, IEEE Journal of Selected Topics in Signal Processing.

[19]  Yo-Sung Ho,et al.  Depth Reconstruction Filter and Down/Up Sampling for Depth Coding in 3-D Video , 2009, IEEE Signal Processing Letters.

[20]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[21]  Sam Kwong,et al.  A Rate-Distortion Optimization Algorithm for Rate Control in H.264 , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[22]  Ahmet M. Kondoz,et al.  Sensitivity Analysis of the Human Visual System for Depth Cues in Stereoscopic 3-D Displays , 2011, IEEE Transactions on Multimedia.

[23]  Yasuhiro Takishima,et al.  A study on rate distortion optimization scheme for JVT coder , 2003, Visual Communications and Image Processing.

[24]  Kwanghoon Sohn,et al.  3D JBU based depth video filtering for temporal fluctuation reduction , 2010, 2010 IEEE International Conference on Image Processing.

[25]  Jin Young Lee,et al.  A Fast and Efficient Multi-View Depth Image Coding Method Based on Temporal and Inter-View Correlations of Texture Images , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[26]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[27]  Stewart Worrall,et al.  Reduced resolution depth coding for stereoscopic 3D video , 2010, IEEE Transactions on Consumer Electronics.

[28]  Warnakulasuriya Anil Chandana Fernando,et al.  A new mode selection technique for coding Depth maps of 3D video , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[29]  Houqiang Li,et al.  Joint multiview video plus depth coding , 2010, 2010 IEEE International Conference on Image Processing.

[30]  Antonio Ortega,et al.  Depth map coding with distortion estimation of rendered view , 2010, Electronic Imaging.

[31]  Yücel Altunbasak,et al.  Frame bit allocation for the H.264/AVC video coder via Cauchy-density-based rate and distortion models , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Warnakulasuriya Anil Chandana Fernando,et al.  Display Dependent Preprocessing of Depth Maps Based on Just Noticeable Depth Difference Modeling , 2011, IEEE Journal of Selected Topics in Signal Processing.

[33]  Zhaoyang Lu,et al.  Model-Based Joint Bit Allocation Between Texture Videos and Depth Maps for 3-D Video Coding , 2011, IEEE Transactions on Circuits and Systems for Video Technology.