Towards Flexible Blind JPEG Artifacts Removal

Training a single deep blind model to handle different quality factors for JPEG image artifacts removal has been attracting considerable attention due to its convenience for practical usage. However, existing deep blind methods usually directly reconstruct the image without predicting the quality factor, thus lacking the flexibility to control the output as the non-blind methods. To remedy this problem, in this paper, we propose a flexible blind convolutional neural network, namely FBCNN, that can predict the adjustable quality factor to control the trade-off between artifacts removal and details preservation. Specifically, FBCNN decouples the quality factor from the JPEG image via a decoupler module and then embeds the predicted quality factor into the subsequent reconstructor module through a quality factor attention block for flexible control. Besides, We find existing methods are prone to fail on non-aligned double JPEG images even with only one pixel shift, and we thus propose a double JPEG degradation model to augment the training data. Extensive experiments on single JPEG images, more general double JPEG images and real-world JPEG images demonstrate that our proposed FBCNN achieves favorable performance against state-of-the-art methods in terms of both quantitative metrics and visual quality.

[1]  Y.-L. Chen,et al.  Detecting Recompression of JPEG Images via Periodicity Analysis of Compression Artifacts for Tampering Detection , 2011, IEEE Transactions on Information Forensics and Security.

[2]  Yun Fu,et al.  Residual Non-local Attention Networks for Image Restoration , 2019, ICLR.

[3]  Shiguang Shan,et al.  AttGAN: Facial Attribute Editing by Only Changing What You Want , 2017, IEEE Transactions on Image Processing.

[4]  Wenhan Yang,et al.  Dmcnn: Dual-Domain Multi-Scale Convolutional Neural Network for Compression Artifacts Removal , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[5]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Manish Okade,et al.  Robust first quantization matrix estimation based on filtering of recompression artifacts for non-aligned double compressed JPEG images , 2018, Signal Process. Image Commun..

[7]  Luc Van Gool,et al.  Designing a Practical Degradation Model for Deep Blind Image Super-Resolution , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Nam Ik Cho,et al.  AGARNet: Adaptively Gated JPEG Compression Artifacts Removal Network for a Wide Range Quality Factor , 2020, IEEE Access.

[9]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[10]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Luca Benini,et al.  CAS-CNN: A deep convolutional neural network for image compression artifact suppression , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[12]  Wei Su,et al.  A generalized Benford's law for JPEG coefficients and its applications in image forensics , 2007, Electronic Imaging.

[13]  Junbin Fang,et al.  An improved parameter estimation scheme for image modification detection based on DCT coefficient analysis. , 2016, Forensic science international.

[14]  Fernando Pérez-González,et al.  Multiple JPEG compression detection by means of Benford-Fourier coefficients , 2014, 2014 IEEE International Workshop on Information Forensics and Security (WIFS).

[15]  Bin Li,et al.  MSE period based estimation of first quantization step in double compressed JPEG images , 2017, Signal Process. Image Commun..

[16]  Xiao Liu,et al.  STGAN: A Unified Selective Transfer Network for Arbitrary Image Attribute Editing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Alessandro Piva,et al.  Analysis of non-aligned double JPEG artifacts for the localization of image forgeries , 2011, 2011 IEEE International Workshop on Information Forensics and Security.

[18]  Wei Wang,et al.  CFSNet: Toward a Controllable Feature Space for Image Restoration , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Fan Zhou,et al.  Implicit Dual-Domain Convolutional Network for Robust Color Image Compression Artifact Reduction , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Sebastiano Battiato,et al.  First Quantization Matrix Estimation From Double Compressed JPEG Images , 2014, IEEE Transactions on Information Forensics and Security.

[21]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[22]  Feng Wu,et al.  JPEG Artifacts Reduction via Deep Convolutional Sparse Coding , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Paolo Bestagini,et al.  Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks , 2017, J. Vis. Commun. Image Represent..

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[26]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[27]  Alessandro Piva,et al.  Image Forgery Localization via Block-Grained Analysis of JPEG Artifacts , 2012, IEEE Transactions on Information Forensics and Security.

[28]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  Mauro Barni,et al.  Identification of cut & paste tampering by means of double-JPEG detection and image segmentation , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[30]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[31]  Lei Zhang,et al.  FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising , 2017, IEEE Transactions on Image Processing.

[32]  Xiaoou Tang,et al.  Compression Artifacts Reduction by a Deep Convolutional Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[33]  Zhe Gan,et al.  AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[35]  Qing Wang,et al.  Double JPEG compression forensics based on a convolutional neural network , 2016, EURASIP J. Inf. Secur..

[36]  Taesung Park,et al.  Semantic Image Synthesis With Spatially-Adaptive Normalization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[39]  Thomas Lukasiewicz,et al.  Controllable Text-to-Image Generation , 2019, NeurIPS.

[40]  Hongyang Chao,et al.  Building Dual-Domain Representations for Compression Artifacts Reduction , 2016, ECCV.

[41]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[43]  Wangmeng Zuo,et al.  Learning a Single Convolutional Super-Resolution Network for Multiple Degradations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Ser-Nam Lim,et al.  Quantization Guided JPEG Artifact Correction , 2020, ECCV.

[45]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Liang Lin,et al.  Multi-level Wavelet-CNN for Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[48]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[49]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[50]  Heung-Kyu Lee,et al.  Double JPEG Detection in Mixed JPEG Quality Factors Using Deep Convolutional Neural Network , 2018, ECCV.

[51]  Xinpeng Zhang,et al.  An improved first quantization matrix estimation for nonaligned double compressed JPEG images , 2020, Signal Process..

[52]  Jiwu Huang,et al.  A Novel Method for Detecting Cropped and Recompressed Image Block , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[53]  Chao Dong,et al.  Interactive Multi-dimension Modulation with Dynamic Controllable Residual Learning for Image Restoration , 2019, ECCV.