Enhancing VVC Through Cnn-Based Post-Processing

This paper presents a new Convolutional Neural Network (CNN) based post-processing approach for video compression, which is applied at the decoder to improve the reconstruction quality. This method has been integrated with the Versatile Video Coding Test Model (VTM) 4.0.1, and evaluated using the Random Access (RA) configuration using the Joint Video Exploration Team (JVET) Common Test Conditions (CTC). The results show coding gains on all tested sequences at various spatial resolutions over different quantisation parameter ranges, with average bit rate savings (based on Bjøntegaard Delta measurements) of 3.90% and 4.13%, when PSNR and VMAF are used as quality metrics respectively. The computational complexities of different CNN architecture variants have also been investigated.

[1]  André Kaup,et al.  Laplace Distribution Based Lagrangian Rate Distortion Optimization for Hybrid Video Coding , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Xinfeng Zhang,et al.  Enhanced Bi-Prediction With Convolutional Neural Network for High-Efficiency Video Coding , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  David Bull,et al.  A Study of High Frame Rate Video Formats , 2019, IEEE Transactions on Multimedia.

[4]  Xinfeng Zhang,et al.  Image and Video Compression With Neural Networks: A Review , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Zulin Wang,et al.  Multi-frame Quality Enhancement for Compressed Video , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Shuai Wan,et al.  Attention-Based Dual-Scale CNN In-Loop Filter for Versatile Video Coding , 2019, IEEE Access.

[8]  Bin Li,et al.  Fully Connected Network-Based Intra Prediction for Image Coding , 2018, IEEE Transactions on Image Processing.

[9]  Dong Liu,et al.  A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding , 2016, MMM.

[10]  Zulin Wang,et al.  Enhancing Quality for HEVC Compressed Videos , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[12]  Angeliki V. Katsenou,et al.  A Subjective Comparison of AV1 and HEVC for Adaptive Video Streaming , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[13]  Chih-Yang Lin,et al.  HEVC Intra Frame Coding Based on Convolutional Neural Network , 2018, IEEE Access.

[14]  Fan Zhang,et al.  ViSTRA2: Video Coding using Spatial Resolution and Effective Bit Depth Adaptation , 2019, ArXiv.

[15]  Mariana Afonso,et al.  Video Compression Based on Spatio-Temporal Resolution Adaptation , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Fan Zhang,et al.  BVI-HD: A Video Quality Database for HEVC Compressed and Texture Synthesized Content , 2018, IEEE Transactions on Multimedia.

[17]  Patrick Le Callet,et al.  CNN-based transform index prediction in multiple transforms framework to assist entropy coding , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[18]  Li Li,et al.  Convolutional Neural Network-Based Fractional-Pixel Motion Compensation , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Mariana Afonso,et al.  Enhanced Video Compression Based on Effective Bit Depth Adaptation , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[20]  Fan Zhang,et al.  A video texture database for perceptual compression and quality assessment , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[21]  Damon M. Chandler,et al.  A perceptual quantization strategy for HEVC based on a convolutional neural network trained on natural images , 2015, SPIE Optical Engineering + Applications.