Dual Discriminator Generative Adversarial Network for Single Image Super-Resolution

Single image super-resolution(SISR) is to reconstruct a high resolution(HR) image from a single low resolution(LR) image. In this paper, with generative adversarial networks(GAN) model as the basic component, we build a dual discriminator super-resolution reconstruction network(DDSRRN) to improve the quality of image super-resolution reconstruction. By adding another discriminator based on GAN, we combine the Kullback-Leibler(KL) with reverse KL divergence to make a unified objective function to train the two discriminators, and by using the complementary statistical characteristics of these two divergences, the prediction density is effectively dispersed in multi-mode, which can avoid collapse of the network model during the reconstruction process and improve the stability of model training. We build the content loss function using the Charbonnier loss and use the intermediate features information of the two discriminators to build the perceptual loss function and style loss function. The experimental results show that the proposed method has sharp edges and rich details in subjective vision, and obtains better subjective visual evaluation and objective quantitative evaluation.

[1]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[2]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[3]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Siyuan Liu,et al.  Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Luc Van Gool,et al.  Regressor Basis Learning for anchored super-resolution , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[6]  Jonathan T. Barron,et al.  A General and Adaptive Robust Loss Function , 2017, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[8]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[9]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[11]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Guangyu Sun,et al.  SRPGAN: Perceptual Generative Adversarial Network for Single Image Super Resolution , 2017, ArXiv.

[13]  Trung Le,et al.  Dual Discriminator Generative Adversarial Nets , 2017, NIPS.

[14]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Yu Qiao,et al.  ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks , 2018, ECCV Workshops.

[18]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[19]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[20]  Ming Qin,et al.  Densely Connected High Order Residual Network for Single Frame Image Super Resolution , 2018, ArXiv.

[21]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Hamid K. Aghajan,et al.  Sensor array processing techniques for super resolution multi-line-fitting and straight edge detection , 1993, IEEE Trans. Image Process..

[23]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.