Coarse-to-Fine Image Super-Resolution Using Convolutional Neural Networks

Convolutional neural networks (CNNs) have been widely applied to computer vision fields due to its excellent performance. CNN-based single image super resolution (SR) methods are also put into practice and outperform previous methods. In this paper, we propose a coarse-to-fine CNN method to boost the existing CNN-based SR methods. We design a cascaded CNN architecture with three stages. The first stage takes the low-resolution (LR) image as the input and outputs a high-resolution (HR) image, then the next stage similarly takes the high-resolution result as the input and produces a finer HR image. Finally, the last stage can obtain the finest HR image. Our architecture is trained as one entire CNN which combines three loss functions to optimize the gradient descent procedure. Experiments on ImageNet-based training samples validates the effectiveness of our method on the public benchmark datasets.

[1]  Xinbo Gao,et al.  Single Image Super-Resolution Using Gaussian Process Regression With Dictionary-Based Sampling and Student- ${t}$ Likelihood , 2017, IEEE Transactions on Image Processing.

[2]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[3]  Jun Yu,et al.  Coupled Deep Autoencoder for Single Image Super-Resolution , 2017, IEEE Transactions on Cybernetics.

[4]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[5]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[9]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[10]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[11]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[15]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[16]  Mário Sarcinelli Filho,et al.  Super-Resolution Image Reconstruction Using Nonparametric Bayesian INLA Approximation , 2012, IEEE Transactions on Image Processing.

[17]  Dacheng Tao,et al.  DCT Regularized Extreme Visual Recovery , 2017, IEEE Transactions on Image Processing.

[18]  Fei Zhou,et al.  Interpolation-Based Image Super-Resolution Using Multisurface Fitting , 2012, IEEE Transactions on Image Processing.

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[21]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[23]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[24]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[25]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[27]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[28]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.