Efficient Depth Intra Frame Coding in 3D-HEVC by Corner Points

To improve the coding performance of depth maps, 3D-HEVC includes several new depth intra coding tools at the expense of increased complexity due to a flexible quadtree Coding Unit/Prediction Unit (CU/PU) partitioning structure and a huge number of intra mode candidates. Compared to natural images, depth maps contain large plain regions surrounded by sharp edges at the object boundaries. Our observation finds that the features proposed in the literature either speed up the CU/PU size decision or intra mode decision and they are also difficult to make proper predictions for CUs/PUs with the multi-directional edges in depth maps. In this work, we reveal that the CUs with multi-directional edges are highly correlated with the distribution of corner points (CPs) in the depth map. CP is proposed as a good feature that can guide to split the CUs with multi-directional edges into smaller units until only single directional edge remains. This smaller unit can then be well predicted by the conventional intra mode. Besides, a fast intra mode decision is also proposed for non-CP PUs, which prunes the conventional HEVC intra modes, skips the depth modeling mode decision, and early determines segment-wise depth coding. Furthermore, a two-step adaptive corner point selection technique is designed to make the proposed algorithm adaptive to frame content and quantization parameters, with the capability of providing the flexible tradeoff between the synthesized view quality and complexity. Simulation results show that the proposed algorithm can provide about 66% time reduction of the 3D-HEVC intra encoder without incurring noticeable performance degradation for synthesized views and it also outperforms the previous state-of-the-art algorithms in term of time reduction and $\Delta $ BDBR.

[1]  Chun-Su Park Efficient intra-mode decision algorithm skipping unnecessary depth-modelling modes in 3D-HEVC , 2015 .

[2]  Ying Chen,et al.  Overview of the Multiview and 3D Extensions of High Efficiency Video Coding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Heiko Schwarz,et al.  3D High-Efficiency Video Coding for Multi-View Video and Depth Data , 2013, IEEE Transactions on Image Processing.

[4]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Yui-Lam Chan,et al.  Fast wedgelet pattern decision for DMM in 3D-HEVC , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[6]  F. Jager Simplified depth map intra coding with an optional depth lookup table , 2012, 2012 International Conference on 3D Imaging (IC3D).

[7]  Gustavo Sanchez,et al.  Block-level fast coding scheme for depth maps in three-dimensional high efficiency video coding , 2018 .

[8]  Heiko Schwarz,et al.  Video Compression Using Nested Quadtree Structures, Leaf Merging, and Improved Techniques for Motion Representation and Entropy Coding , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Yui-Lam Chan,et al.  Probability-Based Depth Intra-Mode Skipping Strategy and Novel VSO Metric for DMM Decision in 3D-HEVC , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Byung Tae Oh,et al.  View Synthesis Distortion Estimation for AVC- and HEVC-Compatible 3-D Video Coding , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Yui-Lam Chan,et al.  Quadtree decision for depth intra coding in 3D-HEVC by good feature , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Yui-Lam Chan,et al.  Adaptive fast intra mode decision of depth map coding by Low Complexity RD-Cost in 3D-HEVC , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[13]  Thomas Wiegand,et al.  3D video coding with depth modeling modes and view synthesis optimization , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[14]  Heiko Schwarz,et al.  3D video coding using the synthesized view distortion change , 2012, 2012 Picture Coding Symposium.

[15]  Béatrice Pesquet-Popescu,et al.  Initialization, Limitation, and Predictive Coding of the Depth and Texture Quadtree in 3D-HEVC , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[17]  Kemal Ugur,et al.  Intra Coding of the HEVC Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Yong Gan,et al.  Fast depth map mode decision based on depth-texture correlation and edge classification for 3D-HEVC , 2017, J. Vis. Commun. Image Represent..

[19]  Luciano Volcan Agostini,et al.  Complexity reduction of depth intra coding for 3D video extension of HEVC , 2014, 2014 IEEE Visual Communications and Image Processing Conference.

[20]  Masayuki Tanimoto,et al.  3D-TV System with Depth-Image-Based Rendering , 2012 .

[21]  Qiuwen Zhang,et al.  Effective early termination algorithm for depth map intra coding in 3D-HEVC , 2014 .

[22]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Chun-Su Park,et al.  Edge-Based Intramode Selection for Depth-Map Coding in 3D-HEVC , 2015, IEEE Transactions on Image Processing.

[24]  Jianhua Zheng,et al.  Fast bi-partition mode selection for 3D HEVC depth intra coding , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[25]  Bruno Zatt,et al.  Complexity reduction for 3D-HEVC depth maps intra-frame prediction using simplified edge detector algorithm , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[26]  César A. M. Marcon,et al.  A reduced computational effort mode-level scheme for 3D-HEVC depth maps intra-frame prediction , 2018, J. Vis. Commun. Image Represent..

[27]  César A. M. Marcon,et al.  Complexity reduction by modes reduction in RD-list for intra-frame prediction in 3D-HEVC depth maps , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[28]  Xu Jian,et al.  Fast Mode Decision Algorithm for Intra Prediction in HEVC , 2014, 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[29]  Yui-Lam Chan,et al.  Efficient depth intra mode decision by reference pixels classification in 3D-HEVC , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[30]  Mei Yu,et al.  Joint processing and fast encoding algorithm for multi-view depth video , 2016, EURASIP J. Image Video Process..

[31]  Wenyi Liu,et al.  Early termination algorithm for the depth modeling mode in three-dimensional extension of high-efficiency video coding , 2016, J. Electronic Imaging.

[32]  Nam Ling,et al.  Fast single depth intra mode decision for depth map coding in 3D-HEVC , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[33]  Thomas Wiegand,et al.  3D video formats and coding methods , 2010, 2010 IEEE International Conference on Image Processing.

[34]  Feng Wu,et al.  Simplified search algorithm for explicit wedgelet signalization mode in 3D-HEVC , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[35]  Mengmeng Zhang,et al.  A fast depth-map wedgelet partitioning scheme for intra prediction in 3D video coding , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).