CNN-Based Intra-Prediction for Lossless HEVC

The paper proposes a novel block-wise prediction paradigm based on Convolutional Neural Networks (CNNs) for lossless video coding. A deep neural network model which follows a multi-resolution design is employed for block-wise prediction. Several contributions are proposed to improve neural network training. A first contribution proposes a novel loss function formulation for an efficient network training based on a new approach for patch selection. Another contribution consists in replacing all HEVC-based angular intra-prediction modes with a CNN-based intra-prediction method, where each angular prediction mode is complemented by a CNN-based prediction mode using a specifically trained model. Another contribution consists in an efficient adaptation of the CNN-based intra-prediction residual for lossless video coding. Experimental results on standard test sequences show that the proposed coding system outperforms the HEVC standard with an average bitrate improvement of around 5%. To our knowledge, the paper is the first to replace all the traditional HEVC-based angular intra-prediction modes with an intra-prediction method based on modern Machine Learning techniques for lossless video coding applications.

[1]  Dong Liu,et al.  Neural network-based arithmetic coding of intra prediction modes in HEVC , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[2]  Siwei Ma,et al.  A Group Variational Transformation Neural Network for Fractional Interpolation of Video Coding , 2018, 2018 Data Compression Conference.

[3]  Wen Gao,et al.  HEVC Lossless Coding and Improvements , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Marta Karczewicz,et al.  Highly Flexible Coding Structures for Next-Generation Video Compression Standard , 2019, 2019 Data Compression Conference (DCC).

[5]  Ionut Schiopu Depth-Map Image Compression Based on Region and Contour Modeling , 2016 .

[6]  Hao Yang,et al.  Low-Complexity CTU Partition Structure Decision and Fast Intra Mode Decision for Versatile Video Coding , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Adrian Munteanu,et al.  Residual-error prediction based on deep learning for lossless image compression , 2018 .

[8]  Adrian Munteanu,et al.  Deep Learning Based Angular Intra-Prediction for Lossless HEVC Video Coding , 2019, 2019 Data Compression Conference (DCC).

[9]  Kiho Choi,et al.  New Video Codec for High-Quality Video Service and Emerging Applications , 2019, 2019 Data Compression Conference (DCC).

[10]  Chao-Yuan Wu,et al.  Video Compression through Image Interpolation , 2018, ECCV.

[11]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Shiqi Wang,et al.  Extended Quad-Tree Partitioning for Future Video Coding , 2019, 2019 Data Compression Conference (DCC).

[14]  Bin Li,et al.  A convolutional neural network-based approach to rate control in HEVC intra coding , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[15]  Tao Zhang,et al.  Convolutional Neural Networks Based Intra Prediction for HEVC , 2017, 2017 Data Compression Conference (DCC).

[16]  Lu Wang,et al.  Quality Enhancement Network via Multi-Reconstruction Recursive Residual Learning for Video Coding , 2019, IEEE Signal Processing Letters.

[17]  Bo Yan,et al.  Deep Residual Network for Enhancing Quality of the Decoded Intra Frames of Hevc , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[18]  Touradj Ebrahimi,et al.  The JPEG 2000 still image compression standard , 2001, IEEE Signal Process. Mag..

[19]  Philippe Hanhart,et al.  Improved Video Coding Techniques for Next Generation Video Coding Standard , 2019, 2019 Data Compression Conference (DCC).

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Adrian Munteanu,et al.  Macro-Pixel Prediction Based on Convolutional Neural Networks for Lossless Compression of Light Field Images , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[22]  Bin Li,et al.  Fully Connected Network-Based Intra Prediction for Image Coding , 2018, IEEE Transactions on Image Processing.

[23]  Minhua Zhou,et al.  HEVC Deblocking Filter , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Munchurl Kim,et al.  CNN-based in-loop filtering for coding efficiency improvement , 2016, 2016 IEEE 12th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP).

[25]  Ke CHEN,et al.  CNN Oriented Fast CU Partition Decision and PU Mode Decision for HEVC Intra Encoding , 2018, 2018 14th IEEE International Conference on Solid-State and Integrated Circuit Technology (ICSICT).

[26]  Fabien Racapé,et al.  CNN-Based Driving of Block Partitioning for Intra Slices Encoding , 2019, 2019 Data Compression Conference (DCC).

[27]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Adrian Munteanu,et al.  Deep-Learning-Based Lossless Image Coding , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Zulin Wang,et al.  Reducing Complexity of HEVC: A Deep Learning Approach , 2017, IEEE Transactions on Image Processing.

[30]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[31]  Guillermo Sapiro,et al.  The LOCO-I lossless image compression algorithm: principles and standardization into JPEG-LS , 2000, IEEE Trans. Image Process..

[32]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[33]  Zulin Wang,et al.  Enhancing Quality for HEVC Compressed Videos , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Dong Liu,et al.  A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding , 2016, MMM.

[35]  Bolei Zhou,et al.  Scene Parsing through ADE20K Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Guangming Shi,et al.  Variable Block-Sized Signal-Dependent Transform for Video Coding , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Yu Liu,et al.  CNN-based Prediction for Lossless Coding of Photographic Images , 2018, 2018 Picture Coding Symposium (PCS).

[38]  D. Marpe,et al.  Neural network based intra prediction for video coding , 2018, Optical Engineering + Applications.

[39]  Heiko Schwarz,et al.  Intra Picture Prediction for Video Coding with Neural Networks , 2019, 2019 Data Compression Conference (DCC).