Deep unsupervised learning for image super-resolution with generative adversarial network

Abstract The aim of Image super-resolution (SR) is to recover high-resolution images from low-resolution ones. By virtue of the great success in numerous computer vision tasks achieved by the convolutional neural networks (CNNs), it is a nice direction to tackle the SR problem using CNNs. Despite progress in accuracy of SR using deeper CNNs, those models are almost trained base upon supervised way. In this paper, we propose a deep unsupervised learning approach for SR with a Generative Adversarial Network (GAN) framework, which is composed of a deep convolutional generator network with dense connections and a discriminator. A sub-pixel convolutional layer is operated on the top of the generator to upscale the inputs, and the standard convolutions are all implemented in the LR space, which leads to a fast restoration. The generator is trained to directly recover the high-resolution image from the low-resolution image. Strided convolution and ReLU activations are employed in the discriminator to distinguish the HR images from the produced HR images. The generator model is optimized with a combination of a data error, a regular term and an adversarial loss, which ensures local–global contents consistency and pixel faithfulness. Note that no labeled training data is employed during the training. Comparisons with several state-of-the-art supervised learn-based methods, experimental results demonstrate that the proposed model achieves a comparable result in terms of both quantitative and qualitative measurements, and it also implies the feasibility and effectiveness of the proposed unsupervised learning-based single-image super-resolution algorithm.