SGCRSR: Sequential gradient constrained regression for single image super-resolution

Abstract Single image super-resolution (SISR), which aims to produce an image with higher resolution and better visual quality from the given single low-resolution (LR) image, has attracted extensive attention in recent years. In particular, the regression-based SISR approaches, which learn the mapping between LR and high-resolution (HR) patch pairs, are efficient and effective as a whole. However, the super-resolved images produced by this kind of method often suffer from visual artifacts as no extra constraints or priors are enforced. To alleviate these shortcomings, we propose a Sequential Gradient Constrained Regression-based single image Super-Resolution (SGCRSR) framework, which provides an effective way to combine the conventional learning-based and reconstruction-based approaches. Firstly, we improve the performance of the well-known super-resolution (SR) method A+ by addressing its deficiencies in both training and testing stages and propose the enhanced A+ (EA+). Then, the EA+ model is applied in dual intensity–gradient domain to construct the Gradient Constrained Regression (GCR)-based SR unit. Finally, a GCR-based sequential SR framework, namely the SGCRSR, is established to improve the quality of super-resolved images gradually. Extensive experiments show that the proposed SGCRSR achieves better performance than several state-of-the-art SR algorithms.

[1]  Wan-Chi Siu,et al.  Fast Image Interpolation via Random Forests , 2015, IEEE Transactions on Image Processing.

[2]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[3]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[4]  Lei Zhang,et al.  Joint Learning of Multiple Regressors for Single Image Super-Resolution , 2016, IEEE Signal Processing Letters.

[5]  Xiaohai He,et al.  Single Image Super-Resolution via Adaptive Transform-Based Nonlocal Self-Similarity Modeling and Learning-Based Gradient Regularization , 2017, IEEE Transactions on Multimedia.

[6]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[7]  Chao Ren,et al.  Single image super resolution using local smoothness and nonlocal self-similarity priors , 2016, Signal Process. Image Commun..

[8]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[9]  Michael Elad,et al.  Multi-Scale Patch-Based Image Restoration , 2016, IEEE Transactions on Image Processing.

[10]  Thomas S. Huang,et al.  Robust Single Image Super-Resolution via Deep Networks With Sparse Prior , 2016, IEEE Transactions on Image Processing.

[11]  Gaofeng Meng,et al.  Edge-Directed Single-Image Super-Resolution Via Adaptive Gradient Magnitude Self-Interpolation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[13]  Moncef Gabbouj,et al.  Image Interpolation Based on Non-local Geometric Similarities and Directional Gradients , 2016, IEEE Trans. Multim..

[14]  Truong Q. Nguyen,et al.  Single Image Super-Resolution Using Local Geometric Duality and Non-Local Similarity , 2016, IEEE Transactions on Image Processing.

[15]  Shiguang Shan,et al.  Deep Network Cascade for Image Super-resolution , 2014, ECCV.

[16]  Shuyuan Yang,et al.  Dual-Geometric Neighbor Embedding for Image Super Resolution With Sparse Tensor , 2014, IEEE Transactions on Image Processing.

[17]  Tao Li,et al.  Rotation expanded dictionary-based single image super-resolution , 2016, Neurocomputing.

[18]  Wenhan Yang,et al.  Isophote-Constrained Autoregressive Model With Adaptive Window Extension for Image Interpolation , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Pingkun Yan,et al.  Image Super-Resolution Via Double Sparsity Regularized Manifold Learning , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Weihong Li,et al.  Multi-layer strategy and reconstruction model with low rank and local rank regularizations for single image super-resolution , 2017, Signal Process. Image Commun..

[21]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Chun Qi,et al.  Position constraint based face image super-resolution by learning multiple local linear projections , 2015, Signal Process. Image Commun..

[23]  Wan-Chi Siu,et al.  Robust Soft-Decision Interpolation Using Weighted Least Squares , 2012, IEEE Transactions on Image Processing.

[24]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[25]  Chunhong Pan,et al.  Fast Image Upsampling via the Displacement Field , 2014, IEEE Transactions on Image Processing.

[26]  Hua Huang,et al.  Neighbor embedding based super-resolution algorithm through edge detection and feature selection , 2009, Pattern Recognit. Lett..

[27]  Fei Zhou,et al.  Consistent Coding Scheme for Single-Image Super-Resolution Via Independent Dictionaries , 2016, IEEE Transactions on Multimedia.

[28]  Yanning Zhang,et al.  Single Image Super-resolution Using Deformable Patches , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Feilong Cao,et al.  Image Interpolation via Low-Rank Matrix Completion and Recovery , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[32]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[33]  Chih-Yuan Yang,et al.  Single-Image Super-Resolution: A Benchmark , 2014, ECCV.

[34]  Michael Elad,et al.  Single Image Interpolation Via Adaptive Nonlocal Sparsity-Based Modeling , 2014, IEEE Transactions on Image Processing.

[35]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[37]  Qionghai Dai,et al.  CCR: Clustering and Collaborative Representation for Fast Single Image Super-Resolution , 2016, IEEE Transactions on Multimedia.

[38]  Srimanta Mandal,et al.  Employing structural and statistical information to learn dictionary(s) for single image super-resolution in sparse domain , 2016, Signal Process. Image Commun..

[39]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[40]  Wei Ye,et al.  Convolutional Edge Diffusion for Fast Contrast-guided Image Interpolation , 2016, IEEE Signal Processing Letters.

[41]  Zhiliang Zhu,et al.  Fast Single Image Super-Resolution via Self-Example Learning and Sparse Representation , 2014, IEEE Transactions on Multimedia.

[42]  Xuelong Li,et al.  Coarse-to-Fine Learning for Single-Image Super-Resolution , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Stanley Osher,et al.  Image Super-Resolution by TV-Regularization and Bregman Iteration , 2008, J. Sci. Comput..

[44]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[45]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[46]  Kai-Kuang Ma,et al.  Contrast-Guided Image Interpolation , 2013, IEEE Transactions on Image Processing.

[47]  Michael Elad,et al.  A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution , 2014, IEEE Transactions on Image Processing.

[48]  Lei Zhang,et al.  Convolutional Sparse Coding for Image Super-Resolution , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[49]  Yong Dou,et al.  Variational single image interpolation with time-varying regularization , 2017, Signal Process. Image Commun..

[50]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[52]  Truong Q. Nguyen,et al.  Single Image Superresolution Based on Gradient Profile Sharpness , 2015, IEEE Transactions on Image Processing.

[53]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[54]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[55]  Xuelong Li,et al.  Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution , 2015, IEEE Transactions on Image Processing.

[56]  Xuelong Li,et al.  SERF: A Simple, Effective, Robust, and Fast Image Super-Resolver From Cascaded Linear Regression , 2016, IEEE Transactions on Image Processing.

[57]  Emmanuel J. Candès,et al.  Super-resolution via Transform-Invariant Group-Sparse Regularization , 2013, 2013 IEEE International Conference on Computer Vision.

[58]  Yi Wan,et al.  Image restoration using structured sparse representation with a novel parametric data-adaptive transformation matrix , 2017, Signal Process. Image Commun..

[59]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.