Learned Image Restoration for VVC Intra Coding

We propose a learned image restoration network as the post-processing module for emerging Versatile Video Coding (VVC) Intra Profile (https://jvet.hhi. fraunhofer.de) based image coding to further improve the reconstructed image quality. The image restoration network is designed using multi-scale spatial priors to effectively alleviate compression artifacts in the decoded images induced by the quantization based lossy compression algorithms. Experimental results demonstrate the performance gains of our proposed post-porcessing network with VVC Intra coding, offering about 6.5% Bjontegaard-Delta Rate (BD-Rate) reduction for YUV 4:4:4 and 12.2% for YUV 4:2:0, against the VVC Intra without our restoration network on the Test Dataset P/M released by the Computer Vision Lab of ETH Zurich, where the distortion is Peak Signal to Noise Ratio (PSNR).

[1]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Kemal Ugur,et al.  Intra Coding of the HEVC Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Lubomir D. Bourdev,et al.  Real-Time Adaptive Image Compression , 2017, ICML.

[4]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Ming Lu,et al.  Learned Quality Enhancement via Multi-Frame Priors for HEVC Compliant Low-Delay Applications , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[6]  Yi Wang,et al.  Scale-Recurrent Network for Deep Image Deblurring , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  David Minnen,et al.  Joint Autoregressive and Hierarchical Priors for Learned Image Compression , 2018, NeurIPS.

[8]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[11]  Xiaoou Tang,et al.  Compression Artifacts Reduction by a Deep Convolutional Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Yao Wang,et al.  End-to-End Learnt Image Compression via Non-Local Attention Optimization and Improved Context Modeling , 2019, IEEE Transactions on Image Processing.