Learning a Single Model With a Wide Range of Quality Factors for JPEG Image Artifacts Removal

Lossy compression brings artifacts into the compressed image and degrades the visual quality. In recent years, many compression artifacts removal methods based on convolutional neural network (CNN) have been developed with great success. However, these methods usually train a model based on one specific value or a small range of quality factors. Obviously, if the test images quality factor does not match to the assumed value range, then degraded performance will be resulted. With this motivation and further consideration of practical usage, a highly robust compression artifacts removal network is proposed in this article. Our proposed network is a single model approach that can be trained for handling a wide range of quality factors while consistently delivering superior or comparable image artifacts removal performance. To demonstrate, we focus on the JPEG compression with quality factors, ranging from 1 to 60. Note that a turnkey success of our proposed network lies in the novel utilization of the quantization tables as part of the training data. Furthermore, it has two branches in parallel—i.e., the restoration branch and the global branch. The former effectively removes the local artifacts, such as ringing artifacts removal. On the other hand, the latter extracts the global features of the entire image that provides highly instrumental image quality improvement, especially effective on dealing with the global artifacts, such as blocking, color shifting. Extensive experimental results performed on color and grayscale images have clearly demonstrated the effectiveness and efficacy of our proposed single-model approach on the removal of compression artifacts from the decoded image.

[1]  Wen Gao,et al.  Reducing Image Compression Artifacts by Structural Sparse Representation and Quantization Constraint Prior , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Licheng Jiao,et al.  Image deblocking via sparse representation , 2012, Signal Process. Image Commun..

[4]  Wenhan Yang,et al.  Dmcnn: Dual-Domain Multi-Scale Convolutional Neural Network for Compression Artifacts Removal , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[5]  Pavel Zemcík,et al.  Compression Artifacts Removal Using Convolutional Neural Networks , 2016, J. WSCG.

[6]  Stuart German,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1988 .

[7]  Luc Van Gool,et al.  NTIRE 2018 Challenge on Single Image Super-Resolution: Methods and Results , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Bo Gao,et al.  Conformal and Low-Rank Sparse Representation for Image Restoration , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Wen Gao,et al.  Compression Artifact Reduction by Overlapped-Block Transform Coefficient Estimation With Block Similarity , 2013, IEEE Transactions on Image Processing.

[10]  Changhoon Yim,et al.  Quality Assessment of Deblocked Images , 2011, IEEE Transactions on Image Processing.

[11]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Nikolas P. Galatsanos,et al.  Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images , 1993, IEEE Trans. Circuits Syst. Video Technol..

[14]  Xianming Liu,et al.  Data-driven sparsity-based restoration of JPEG-compressed images in dual transform-pixel domain , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[17]  Qing Ling,et al.  D3: Deep Dual-Domain Based Fast Restoration of JPEG-Compressed Images , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Hongyang Chao,et al.  One-To-Many Network for Visually Pleasing Compression Artifacts Reduction , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Tao Chen,et al.  Adaptive postfiltering of transform coefficients for the reduction of blocking artifacts , 2001, IEEE Trans. Circuits Syst. Video Technol..

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[24]  Masanori Suganuma,et al.  Attention-Based Adaptive Selection of Operations for Image Restoration in the Presence of Unknown Combined Distortions , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Michael K. Ng,et al.  Reducing Artifacts in JPEG Decompression Via a Learned Dictionary , 2014, IEEE Transactions on Signal Processing.

[28]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  Xiaoou Tang,et al.  Compression Artifacts Reduction by a Deep Convolutional Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[30]  Jae Lim,et al.  Reduction Of Blocking Effects In Image Coding , 1984 .

[31]  William H. Richardson,et al.  Bayesian-Based Iterative Method of Image Restoration , 1972 .

[32]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Hongyang Chao,et al.  Building Dual-Domain Representations for Compression Artifacts Reduction , 2016, ECCV.

[34]  Luca Benini,et al.  CAS-CNN: A deep convolutional neural network for image compression artifact suppression , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[35]  Guangming Shi,et al.  Denoising Prior Driven Deep Neural Network for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[37]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[38]  Tao Wang,et al.  End-to-end text recognition with convolutional neural networks , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[39]  Alan C. Bovik,et al.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms , 2006, IEEE Transactions on Image Processing.

[40]  Avideh Zakhor Iterative procedures for reduction of blocking effects in transform image coding , 1992, IEEE Trans. Circuits Syst. Video Technol..

[41]  Xianming Liu,et al.  Inter-block consistent soft decoding of JPEG images with sparsity and graph-signal smoothness priors , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[42]  Fan Zhou,et al.  Implicit Dual-Domain Convolutional Network for Robust Color Image Compression Artifact Reduction , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[44]  Yun Fu,et al.  Residual Non-local Attention Networks for Image Restoration , 2019, ICLR.

[45]  Alberto Del Bimbo,et al.  Deep Generative Adversarial Compression Artifact Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[46]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[47]  Touradj Ebrahimi,et al.  The JPEG 2000 still image compression standard , 2001, IEEE Signal Process. Mag..

[48]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.