Progressive multi-scale attention network for compression artifact reduction

Abstract. The recent convolutional neural network based studies on the compression artifact reduction (CAR) task have made great progress. However, most of these CAR methods still have some inadequacies. They are limited on the network capability due to treating extracted features equally and generate unpleasant visual results due to using the pixel-wise loss (e.g., L1/L2 loss) in training. Therefore, to address these issues, we propose a progressive multi-scale attention network (PMANet) for image CAR task and further introduce a PMANet-based generative adversarial network (PMAGAN) for visual quality improvement. Specifically, the key idea and the basic component of the PMANet is a multi-scale attention dense block, which effectively incorporates the multi-scale information to the model with the attention mechanism and thus enhances the network’s representation ability. The PMANet can be further improved with the designed progressive restoration structure. In addition, PMAGAN takes the PMANet as the generator and brings a generative adversarial networks framework with the adversarial training strategy. Experiments show that PMANet performs better than the state-of-the-art CAR methods, and PMAGAN can further achieve better visual quality with more natural and sharper textures.

[1]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Truong Q. Nguyen,et al.  Blocking artifact free inverse discrete cosine transform , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[3]  Rui Sun,et al.  S-Net: a scalable convolutional neural network for JPEG compression artifact reduction , 2018, J. Electronic Imaging.

[4]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[5]  Yun Fu,et al.  Residual Non-local Attention Networks for Image Restoration , 2019, ICLR.

[6]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[8]  Yunjin Chen,et al.  Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Michael K. Ng,et al.  Reducing Artifacts in JPEG Decompression Via a Learned Dictionary , 2014, IEEE Transactions on Signal Processing.

[10]  Hongyang Chao,et al.  One-To-Many Network for Visually Pleasing Compression Artifacts Reduction , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Luca Benini,et al.  CAS-CNN: A deep convolutional neural network for image compression artifact suppression , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[12]  Alexia Jolicoeur-Martineau,et al.  The relativistic discriminator: a key element missing from standard GAN , 2018, ICLR.

[13]  Xiaoou Tang,et al.  Compression Artifacts Reduction by a Deep Convolutional Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Thomas S. Huang,et al.  Wide-activated Deep Residual Networks based Restoration for BPG-compressed Images , 2018, CVPR Workshops.

[15]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[18]  Karen O. Egiazarian,et al.  Pointwise Shape-Adaptive DCT for High-Quality Denoising and Deblocking of Grayscale and Color Images , 2007, IEEE Transactions on Image Processing.

[19]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[21]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[22]  Fan Zhou,et al.  Implicit Dual-Domain Convolutional Network for Robust Color Image Compression Artifact Reduction , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[24]  Yun Fu,et al.  Residual Dense Network for Image Restoration , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[27]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[28]  Hongyang Chao,et al.  Building Dual-Domain Representations for Compression Artifacts Reduction , 2016, ECCV.

[29]  Peng Gao,et al.  Accurate and Efficient Image Super-Resolution via Global-Local Adjusting Dense Network , 2021, IEEE Transactions on Multimedia.

[30]  Alberto Del Bimbo,et al.  Deep Generative Adversarial Compression Artifact Removal , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Changhoon Yim,et al.  Quality Assessment of Deblocked Images , 2011, IEEE Transactions on Image Processing.