Using Deep Learning Neural Network for Block Partitioning in H.265/HEVC

dividing video frames into Coding Tree Units (CTUs) and Coding Units (CUs) is a critical task of video compression in H.265/HEVC video coding standard. In this paper, we utilize deep learning techniques, especially the deep Convolutional Neural Network (CNN) to speed up the block partitioning process. Deep CNNs have achieved break-through improvements on image recognition tasks such as image classifications, object identifications, and image annotations. However, very few work has been done in applying deep CNN to video encoding. Block partitioning in video coding is highly dependent on the content of the video frames, and thus it is natural to take advantage of the significant capabilities of deep CNN on image content detection and recognition to perform block partitioning and avoid the time-consuming iterative RateDistortion-Optimization (RDO) process. Experimental results have shown that the proposed methodology has largely speed up the coding process and has also achieved coding efficiency comparable to the reference software of H.265/HEVC.

[1]  Pao-Chi Chang,et al.  Machine Learning-based Fast Intra Coding Unit Depth Decision for High Efficiency Video Coding , 2016, J. Inf. Sci. Eng..

[2]  Zhan Ma,et al.  Fast CU partition decision using machine learning for screen content compression , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[3]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Dongsheng Wang,et al.  CNN oriented fast HEVC intra CU mode decision , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[5]  Nuno Roma,et al.  Run-Time Machine Learning for HEVC/H.265 Fast Partitioning Decision , 2015, 2015 IEEE International Symposium on Multimedia (ISM).

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Zhenyu Liu,et al.  CU Partition Mode Decision for HEVC Hardwired Intra Encoder Using Convolution Neural Network , 2016, IEEE Transactions on Image Processing.

[8]  Damon M. Chandler,et al.  A perceptual quantization strategy for HEVC based on a convolutional neural network trained on natural images , 2015, SPIE Optical Engineering + Applications.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Gary J. Sullivan,et al.  Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC) , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).