Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means

The goal of learning-based image super resolution (SR) is to generate a plausible and visually pleasing high-resolution (HR) image from a given low-resolution (LR) input. The SR problem is severely underconstrained, and it has to rely on examples or some strong image priors to reconstruct the missing HR image details. This paper addresses the problem of learning the mapping functions (i.e., projection matrices) between the LR and HR images based on a dictionary of LR and HR examples. Encouraged by recent developments in image prior modeling, where the state-of-the-art algorithms are formed with nonlocal self-similarity and local geometry priors, we seek an SR algorithm of similar nature that will incorporate these two priors into the learning from LR space to HR space. The nonlocal self-similarity prior takes advantage of the redundancy of similar patches in natural images, while the local geometry prior of the data space can be used to regularize the modeling of the nonlinear relationship between LR and HR spaces. Based on the above two considerations, we first apply the local geometry prior to regularize the patch representation, and then utilize the nonlocal means filter to improve the super-resolved outcome. Experimental results verify the effectiveness of the proposed algorithm compared with the state-of-the-art SR methods.

[1]  Jie Ren,et al.  Context-Aware Sparse Decomposition for Image Denoising and Super-Resolution , 2013, IEEE Transactions on Image Processing.

[2]  Ruimin Hu,et al.  Locally regularized Anchored Neighborhood Regression for fast Super-Resolution , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[3]  Michael Elad,et al.  Superresolution restoration of an image sequence: adaptive filtering approach , 1999, IEEE Trans. Image Process..

[4]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[5]  Ruimin Hu,et al.  Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation , 2016, Inf. Sci..

[6]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[7]  Zhiliang Zhu,et al.  Fast Single Image Super-Resolution via Self-Example Learning and Sparse Representation , 2014, IEEE Transactions on Multimedia.

[8]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[10]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Ruimin Hu,et al.  Noise Robust Face Hallucination via Locality-Constrained Representation , 2014, IEEE Transactions on Multimedia.

[12]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[13]  Yuan Yan Tang,et al.  Weighted Joint Sparse Representation for Removing Mixed Noise in Image , 2017, IEEE Transactions on Cybernetics.

[14]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[16]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[17]  Ke Lu,et al.  Compressed Sensing of a Remote Sensing Image Based on the Priors of the Reference Image , 2015, IEEE Geoscience and Remote Sensing Letters.

[18]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[19]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[20]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[21]  Xuelong Li,et al.  Joint Learning for Single-Image Super-Resolution via a Coupled Constraint , 2012, IEEE Transactions on Image Processing.

[22]  Lizhe Wang,et al.  Reference Information Based Remote Sensing Image Reconstruction with Generalized Nonconvex Low-Rank Approximation , 2016, Remote. Sens..

[23]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[24]  Dacheng Tao,et al.  Single Image Superresolution via Directional Group Sparsity and Directional Features , 2015, IEEE Transactions on Image Processing.

[25]  Yuan Yan Tang,et al.  Weighted Couple Sparse Representation With Classified Regularization for Impulse Noise Removal , 2015, IEEE Transactions on Image Processing.

[26]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[28]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[29]  Kun Li,et al.  Video super-resolution using an adaptive superpixel-guided auto-regressive model , 2016, Pattern Recognit..

[30]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Xuelong Li,et al.  SERF: A Simple, Effective, Robust, and Fast Image Super-Resolver From Cascaded Linear Regression , 2016, IEEE Transactions on Image Processing.

[32]  Xiang Ma,et al.  Sparse Support Regression for Image Super-Resolution , 2015, IEEE Photonics Journal.

[33]  Hayit Greenspan,et al.  Super-Resolution in Medical Imaging , 2009, Comput. J..

[34]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[35]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[36]  H Stark,et al.  High-resolution image recovery from image-plane arrays, using convex projections. , 1989, Journal of the Optical Society of America. A, Optics and image science.

[37]  Klaus Diepold,et al.  Analysis Operator Learning and its Application to Image Reconstruction , 2012, IEEE Transactions on Image Processing.

[38]  Yihong Gong,et al.  Nonlinear Learning using Local Coordinate Coding , 2009, NIPS.

[39]  Kun Li,et al.  Video super-resolution based on automatic key-frame selection and feature-guided variational optical flow , 2014, Signal Process. Image Commun..

[40]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[41]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Lizhe Wang,et al.  Fast and Scalable Multi-Way Analysis of Massive Neural Data , 2015, IEEE Transactions on Computers.

[43]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[44]  Wenhan Yang,et al.  Image Super-Resolution Based on Structure-Modulated Sparse Representation , 2015, IEEE Transactions on Image Processing.

[45]  Dong Xu,et al.  Example-Based Super-Resolution With Soft Information and Decision , 2013, IEEE Transactions on Multimedia.

[46]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[47]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[48]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  Chen Chen,et al.  Single-image super-resolution using multihypothesis prediction , 2012, 2012 Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[50]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[51]  Guangming Shi,et al.  Sparsity Fine Tuning in Wavelet Domain With Application to Compressive Image Reconstruction , 2014, IEEE Transactions on Image Processing.

[52]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[53]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[54]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[55]  Alan L. Yuille,et al.  Non-Rigid Point Set Registration by Preserving Global and Local Structures , 2016, IEEE Transactions on Image Processing.

[56]  Albert Y. Zomaya,et al.  Parallel Simulation of Complex Evacuation Scenarios with Adaptive Agent Models , 2015, IEEE Transactions on Parallel and Distributed Systems.

[57]  Stéphane Mallat,et al.  Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity , 2010, IEEE Transactions on Image Processing.

[58]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[59]  Guillermo Sapiro,et al.  A Variational Framework for Non-local Image Inpainting , 2009, EMMCVPR.

[60]  Yu-Chiang Frank Wang,et al.  A Self-Learning Approach to Single Image Super-Resolution , 2013, IEEE Transactions on Multimedia.