Deep Differential Convolutional Network for Single Image Super-Resolution

The deep convolutional neural networks and residual networks have shown great success and high-quality reconstruction for single image super-resolution. It is clearly seen that among the best-known super-resolution models, deep learning-based methods demonstrate state-of-the-art performance. In this paper, we propose a deep differential convolutional network (DCN) for single image super-resolution (SRDCN). The proposed DCN is a novel convolutional network, which is composed of convolutional layers, parametric rectified linear units (PReLU), and the identity skip connection. Different from other deep learning-based methods which complete the reconstruction by learning the mapping function between low-resolution and high-resolution images, the proposed algorithm makes changes to the way of reconstruction. In the proposed network, we use DCN to obtain the reconstructed images and the differences between the low-resolution and reconstructed images in the reconstruction process. Then the differences combined with the original low-resolution image and the reconstructed image that from the last DCN are used for final reconstruction. In addition, the loss function is more rationally designed and optimized in this paper. The proposed loss function contains three parts of loss: feature loss, style loss, and mean squared error (MSE) loss. These losses will be used to supervise the structure and content of the reconstructed image. The experimental results prove that the proposed model is superior to many state-of-the-art super-resolution methods in terms of both peak signal-to-noise ratio (PSNR) and structural similarity index metrics (SSIM).

[1]  Dacheng Tao,et al.  Single Image Superresolution via Directional Group Sparsity and Directional Features , 2015, IEEE Transactions on Image Processing.

[2]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[3]  Lisimachos P. Kondi,et al.  A regularization framework for joint blur estimation and super-resolution of video sequences , 2005, IEEE International Conference on Image Processing 2005.

[4]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[6]  Joan Bruna,et al.  Super-Resolution with Deep Convolutional Sufficient Statistics , 2015, ICLR.

[7]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[8]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yang Zhao,et al.  Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000 , 2018 .

[10]  Leon A. Gatys,et al.  A Neural Algorithm of Artistic Style , 2015, ArXiv.

[11]  Martin Johnston,et al.  Variational Bayesian Subgroup Adaptive Sparse Component Extraction for Diagnostic Imaging System , 2018, IEEE Transactions on Industrial Electronics.

[12]  Oscar C. Au,et al.  Graph-based joint denoising and super-resolution of generalized piecewise smooth images , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[13]  Liangpei Zhang,et al.  A super-resolution reconstruction algorithm for surveillance images , 2010, Signal Process..

[14]  Nikolas P. Galatsanos,et al.  Maximum a Posteriori Video Super-Resolution Using a New Multichannel Image Prior , 2010, IEEE Transactions on Image Processing.

[15]  Yücel Altunbasak,et al.  Eigenface-domain super-resolution for face recognition , 2003, IEEE Trans. Image Process..

[16]  Thomas Brox,et al.  Generating Images with Perceptual Similarity Metrics based on Deep Networks , 2016, NIPS.

[17]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[18]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[19]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[20]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[21]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[23]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[24]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[25]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[26]  Luc Van Gool,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[30]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Peter M. Atkinson,et al.  Sub‐pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super‐resolution pixel‐swapping , 2006 .

[32]  Daniel Rueckert,et al.  Cardiac Image Super-Resolution with Global Correspondence Using Multi-Atlas PatchMatch , 2013, MICCAI.

[33]  Michael Elad,et al.  A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution , 2014, IEEE Transactions on Image Processing.

[34]  S Peled,et al.  Superresolution in MRI: Application to human white matter fiber tract visualization by diffusion tensor imaging , 2001, Magnetic resonance in medicine.

[35]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[38]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[39]  Liang Wang,et al.  Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution , 2015, NIPS.

[40]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[41]  Xuelong Li,et al.  A multi-frame image super-resolution method , 2010, Signal Process..

[42]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[44]  Horst Bischof,et al.  Fast and accurate image upscaling with super-resolution forests , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[46]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[47]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[48]  Yan Liu,et al.  Multi-Branch Deep Residual Network for Single Image Super-Resolution , 2018, Algorithms.

[49]  Hua Huang,et al.  Neighbor embedding based super-resolution algorithm through edge detection and feature selection , 2009, Pattern Recognit. Lett..

[50]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Michael Elad,et al.  Robust shift and add approach to superresolution , 2003, SPIE Optics + Photonics.

[52]  Jun Cheng,et al.  Superpixel-guided nonlocal means for image denoising and super-resolution , 2016, Signal Process..