Local patch encoding-based method for single image super-resolution

Abstract Recent learning-based super-resolution (SR) methods often focus on dictionary learning or network training. In this paper, we discuss in detail a new SR method based on local patch encoding (LPE) instead of traditional dictionary learning. The proposed method consists of a learning stage and a reconstructing stage. In the learning stage, image patches are classified into different classes by means of the proposed LPE, and then a projection matrix is computed for each class by utilizing a simple constraint. In the reconstructing stage, an input LR patch can be simply reconstructed by computing its LPE code and then multiplying the corresponding projection matrix. Furthermore, we discuss the relationship between the proposed method and the anchored neighborhood regression methods; we also analyze the extendibility of the proposed method. The experimental results on several image sets demonstrate the effectiveness of the LPE-based methods.

[1]  Thomas Martin Deserno,et al.  Survey: interpolation methods in medical image processing , 1999, IEEE Transactions on Medical Imaging.

[2]  Chih-Yuan Yang,et al.  Single-Image Super-Resolution: A Benchmark , 2014, ECCV.

[3]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[4]  Peyman Milanfar,et al.  RAISR: Rapid and Accurate Image Super Resolution , 2016, IEEE Transactions on Computational Imaging.

[5]  David H. Frakes,et al.  Segment Adaptive Gradient Angle Interpolation , 2013, IEEE Transactions on Image Processing.

[6]  Cem Yuksel,et al.  Dual scattering approximation for fast multiple scattering in hair , 2008, SIGGRAPH 2008.

[7]  M. Tech,et al.  Real Time Artifact-Free Image Upscaling , 2012 .

[8]  Raanan Fattal,et al.  Image and video upscaling from local self-examples , 2011, TOGS.

[9]  Dong Xu,et al.  Example-Based Super-Resolution With Soft Information and Decision , 2013, IEEE Transactions on Multimedia.

[10]  Ruimin Hu,et al.  Noise robust position-patch based face super-resolution via Tikhonov regularized neighbor representation , 2016, Inf. Sci..

[11]  Gustavo de Veciana,et al.  An information fidelity criterion for image quality assessment using natural scene statistics , 2005, IEEE Transactions on Image Processing.

[12]  Michal Irani,et al.  Nonparametric Blind Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[13]  Weiguo Gong,et al.  Combining sparse representation and local rank constraint for single image super resolution , 2015, Inf. Sci..

[14]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[15]  Yu Zhou,et al.  A phase congruency based patch evaluator for complexity reduction in multi-dictionary based single-image super-resolution , 2016, Inf. Sci..

[16]  Ting-Zhu Huang,et al.  Single image super-resolution by approximated Heaviside functions , 2015, Inf. Sci..

[17]  Fei Zhou,et al.  Interpolation-Based Image Super-Resolution Using Multisurface Fitting , 2012, IEEE Transactions on Image Processing.

[18]  Chunhong Pan,et al.  Fast Image Upsampling via the Displacement Field , 2014, IEEE Transactions on Image Processing.

[19]  Guangtao Zhai,et al.  Single Image Super-resolution With Detail Enhancement Based on Local Fractal Analysis of Gradient , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .

[21]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[22]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[25]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Rabab Kreidieh Ward,et al.  Contrast enhancement for enlarged images based on edge sharpening , 2005, IEEE International Conference on Image Processing 2005.

[27]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Chi-Keung Tang,et al.  Fast image/video upsampling , 2008, SIGGRAPH 2008.

[29]  Gaofeng Meng,et al.  Edge-Directed Single-Image Super-Resolution Via Adaptive Gradient Magnitude Self-Interpolation , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Yu-Chiang Frank Wang,et al.  A Self-Learning Approach to Single Image Super-Resolution , 2013, IEEE Transactions on Multimedia.

[31]  Fei Zhou,et al.  Consistent Coding Scheme for Single-Image Super-Resolution Via Independent Dictionaries , 2016, IEEE Transactions on Multimedia.

[32]  Dacheng Tao,et al.  DCT Regularized Extreme Visual Recovery , 2017, IEEE Transactions on Image Processing.

[33]  Yang Wang,et al.  Single image super-resolution reconstruction based on genetic algorithm and regularization prior model , 2016, Inf. Sci..

[34]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[35]  Liang Wang,et al.  Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution , 2015, NIPS.

[36]  Wen Gao,et al.  Image interpolation via regularized local linear regression , 2011, 28th Picture Coding Symposium.

[37]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[38]  Rabab Kreidieh Ward,et al.  A New Orientation-Adaptive Interpolation Method , 2007, IEEE Transactions on Image Processing.

[39]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[40]  Gwanggil Jeon,et al.  From coarse- to fine-grained implementation of edge-directed interpolation using a GPU , 2017, Inf. Sci..

[41]  Xuelong Li,et al.  Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution , 2015, IEEE Transactions on Image Processing.

[42]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[43]  Anat Levin,et al.  Accurate Blur Models vs. Image Priors in Single Image Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[44]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[45]  Liujuan Cao,et al.  Vehicle detection from highway satellite images via transfer learning , 2016, Inf. Sci..

[46]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Stanley Osher,et al.  Image Super-Resolution by TV-Regularization and Bregman Iteration , 2008, J. Sci. Comput..

[48]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[50]  Dacheng Tao,et al.  Beyond RPCA: Flattening Complex Noise in the Frequency Domain , 2017, AAAI.

[51]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[52]  Nanning Zheng,et al.  Image hallucination with primal sketch priors , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[53]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[54]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[55]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[56]  Weiguo Gong,et al.  Dual-sparsity regularized sparse representation for single image super-resolution , 2015, Inf. Sci..

[57]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Qionghai Dai,et al.  CCR: Clustering and Collaborative Representation for Fast Single Image Super-Resolution , 2016, IEEE Transactions on Multimedia.

[59]  Horst Bischof,et al.  Fast and accurate image upscaling with super-resolution forests , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Huifang Sun,et al.  Image interpolation via graph-based Bayesian label propagation. , 2014, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.