Allowable depth distortion based fast mode decision and reference frame selection for 3D depth coding

To improve the coding efficiency and meanwhile reduce the complexity of the depth video encoder in 3D system, we present an efficient depth coding scheme based on the allowable depth distortion (ADD) in view synthesis. Firstly, we analyze the depth distortion in view synthesis and present a new ADD-based Rate Distortion (RD) optimization cost function to improve the mode/reference decision criteria. Then, we present an early mode and reference frame selection algorithm, which skips the unnecessary and complex mode and reference frame when the depth distortion is within the ADD interval. Meanwhile, the coding efficiency is improved by RD optimized mode decision and reference frame selection. The experimental results show that the proposed overall algorithm, which integrates the fast mode decision, reference frame selection and RD optimization, can reduce the depth coding complexity from 32.47 to 72.59, and 55.95 % on average over different test sequences. Meanwhile, the proposed algorithm achieves 1.34 and 1.58 dB average Bjontegaard Delta peak signal-to-noise ratio gain in terms of the view synthesis quality at low and high bit rate, respectively. In terms of the bit rate reduction, it reduces 54.66 and 49.60 % Bjontegaard Delta bit rate on average at low and high bit rate, respectively, while maintaining the same view synthesis image quality.

[1]  Zhaoyang Lu,et al.  Model-Based Joint Bit Allocation Between Texture Videos and Depth Maps for 3-D Video Coding , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  C.-C. Jay Kuo,et al.  Subjective and Objective Video Quality Assessment of 3D Synthesized Views With Texture/Depth Compression Distortion , 2015, IEEE Transactions on Image Processing.

[3]  Houqiang Li,et al.  Multiview-Video-Plus-Depth Coding Based on the Advanced Video Coding Standard , 2013, IEEE Transactions on Image Processing.

[4]  C.-C. Jay Kuo,et al.  Efficient Multiview Depth Coding Optimization Based on Allowable Depth Distortion in View Synthesis , 2014, IEEE Transactions on Image Processing.

[5]  Ying Chen,et al.  Overview of the MVC + D 3D video coding standard , 2014, J. Vis. Commun. Image Represent..

[6]  Gangyi Jiang,et al.  Regional Bit Allocation and Rate Distortion Optimization for Multiview Depth Video Coding With View Synthesis Distortion Model , 2013, IEEE Transactions on Image Processing.

[7]  Yo-Sung Ho,et al.  Depth Coding Using a Boundary Reconstruction Filter for 3-D Video Systems , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Heiko Schwarz,et al.  3D video coding using advanced prediction, depth modeling, and encoder control methods , 2012, 2012 Picture Coding Symposium.

[9]  Yo-Sung Ho,et al.  Depth Video Coding Using Adaptive Geometry Based Intra Prediction for 3-D Video Systems , 2012, IEEE Transactions on Multimedia.

[10]  Jing Zhang,et al.  Depth map super-resolution using stereo-vision-assisted model , 2015, Neurocomputing.

[11]  Detlev Marpe,et al.  Depth Intra Coding for 3D Video Based on Geometric Primitives , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Marco Zamarin,et al.  Edge-preserving Intra mode for efficient depth map coding based on H.264/AVC , 2014, Signal Process. Image Commun..

[13]  Yun Zhang,et al.  View synthesis distortion model based frame level rate control optimization for multiview depth video coding , 2015, Signal Process..

[14]  Gangyi Jiang,et al.  Efficient Multi-Reference Frame Selection Algorithm for Hierarchical B Pictures in Multiview Video Coding , 2011, IEEE Transactions on Broadcasting.

[15]  张云 Fast mode decision based on texture-depth correlation and motion prediction for multiview depth video coding , 2013 .

[16]  Sam Kwong,et al.  Efficient Motion and Disparity Estimation Optimization for Low Complexity Multiview Video Coding , 2015, IEEE Transactions on Broadcasting.

[17]  C.-C. Jay Kuo,et al.  Rate-Distortion Optimized Rate Control for Depth Map-Based 3-D Video Coding , 2013, IEEE Transactions on Image Processing.

[18]  Yo-Sung Ho,et al.  Fast Mode Decision Algorithm for Depth Coding in 3D Video Systems Using H.264/AVC , 2011, PSIVT.

[19]  Da-Hyun Yoon,et al.  Fast Mode Decision Algorithm for Depth Coding in 3 D Video Systems Using H . 264 / AVC , 2011 .

[20]  Jing Zhang,et al.  Depth map super-resolution via local and nonlocal priors , 2014, J. Electronic Imaging.

[21]  Linwei Zhu,et al.  View synthesis distortion elimination filter for depth video coding in 3D video broadcasting , 2014, Multimedia Tools and Applications.

[22]  Yue Gao,et al.  Cross-View Down/Up-Sampling Method for Multiview Depth Video Coding , 2012, IEEE Signal Processing Letters.

[23]  Zhenzhong Chen,et al.  Depth No-Synthesis-Error Model for View Synthesis in 3-D Video , 2011, IEEE Transactions on Image Processing.

[24]  Jin Young Lee,et al.  A Fast and Efficient Multi-View Depth Image Coding Method Based on Temporal and Inter-View Correlations of Texture Images , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Gangyi Jiang,et al.  DIRECT Mode Early Decision Optimization Based on Rate Distortion Cost Property and Inter-view Correlation , 2013, IEEE Transactions on Broadcasting.

[26]  Hao Liu,et al.  Dense depth image synthesis via energy minimization for three-dimensional video , 2015, Signal Process..

[27]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[28]  Mei Yu,et al.  Fast macroblock mode selection algorithm for multiview depth video coding , 2010 .

[29]  Byung Tae Oh,et al.  View Synthesis Distortion Estimation for AVC- and HEVC-Compatible 3-D Video Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Beatrice Pesquet-Popescu,et al.  Depth video coding based on intra mode inheritance from texture , 2014, APSIPA Transactions on Signal and Information Processing.