A compendious study of super-resolution techniques by single image

Abstract Single-image Super-resolution (SISR) is the process of reconstructing a high-resolution (HR) image by artificially creating the information and frequency details from a single available low resolution (LR) image. Availability of limited number of images in the practical circumstances has motivated researchers toward this area. Plethora of techniques has been proposed over the years for SR reconstruction. In this paper, the SR process has been classified and the degradation model has been discussed. It also comprises the extensive survey of SISR techniques, including the latest developments in this field. Lastly, it covers the limitations and problems of existing techniques of SISR.

[1]  Thomas Martin Deserno,et al.  Survey: interpolation methods in medical image processing , 1999, IEEE Transactions on Medical Imaging.

[2]  Zhe L. Lin,et al.  Fast Image Super-Resolution Based on In-Place Example Regression , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Michael Elad,et al.  Super-Resolution Reconstruction of Image Sequences , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Frédéric Guichard,et al.  A partial differential equation approach to image zoom , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[5]  Martin Vetterli,et al.  Locally adaptive wavelet-based image interpolation , 2006, IEEE Transactions on Image Processing.

[6]  Wan-Chi Siu,et al.  Robust Soft-Decision Interpolation Using Weighted Least Squares , 2012, IEEE Transactions on Image Processing.

[7]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Yiguang Chen,et al.  Single-Image Super-Resolution Reconstruction via Learned Geometric Dictionaries and Clustered Sparse Coding , 2012, IEEE Transactions on Image Processing.

[9]  Jae-Seok Choi,et al.  Single Image Super-Resolution Using Global Regression Based on Multiple Local Linear Mappings , 2017, IEEE Transactions on Image Processing.

[10]  Qingchun Chen,et al.  Single Image Super-Resolution Reconstruction Technique based on A Single Hybrid Dictionary , 2016, Multimedia Tools and Applications.

[11]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[12]  Jian-Huang Lai,et al.  Face Hallucination under an Image Decomposition Perspective , 2010, 2010 20th International Conference on Pattern Recognition.

[13]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[14]  Jie Ren,et al.  Context-Aware Sparse Decomposition for Image Denoising and Super-Resolution , 2013, IEEE Transactions on Image Processing.

[15]  Truong Q. Nguyen,et al.  Markov Random Field Model-Based Edge-Directed Image Interpolation , 2007, IEEE Transactions on Image Processing.

[16]  Joonki Paik,et al.  Multi-frame example-based super-resolution using locally directional self-similarity , 2015, IEEE Transactions on Consumer Electronics.

[17]  Weidong Yan,et al.  Super-resolution from a single image based on local self-similarity , 2015, Multimedia Tools and Applications.

[18]  Shenghuo Zhu,et al.  Resolution-invariant coding for continuous image super-resolution , 2012, Neurocomputing.

[19]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Karen O. Egiazarian,et al.  Single Image Super-Resolution Based on Wiener Filter in Similarity Domain , 2017, IEEE Transactions on Image Processing.

[21]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[22]  Dahua Lin,et al.  Hallucinating faces: TensorPatch super-resolution and coupled residue compensation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Seongjai Kim,et al.  Edge-forming methods for color image zooming , 2006, IEEE Transactions on Image Processing.

[24]  Wan-Chi Siu,et al.  Learning Hierarchical Decision Trees for Single-Image Super-Resolution , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Naveed Iqbal,et al.  A Bayesian Approach to Face Hallucination Using DLPP and KRR , 2010, 2010 20th International Conference on Pattern Recognition.

[26]  Nirmal K. Bose,et al.  Simultaneous noise filtering and super-resolution with second-generation wavelets , 2005, IEEE Signal Processing Letters.

[27]  Ning Wu,et al.  Fast Facial Image Super-Resolution via Local Linear Transformations for Resource-Limited Applications , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[29]  Ruimin Hu,et al.  Face Hallucination Via Weighted Adaptive Sparse Regularization , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Yanbing Xue,et al.  Super-Resolution Image Reconstruction Based on K-Means-Markov Network , 2009, 2009 Fifth International Conference on Natural Computation.

[31]  Stephen E. Reichenbach,et al.  Image interpolation by two-dimensional parametric cubic convolution , 2006, IEEE Transactions on Image Processing.

[32]  Michal Irani,et al.  Nonparametric Blind Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[33]  Aline Roumy,et al.  Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Wan-Chi Siu,et al.  Image super-resolution via weighted random forest , 2017, 2017 IEEE International Conference on Industrial Technology (ICIT).

[35]  Chunhong Pan,et al.  Fast Image Upsampling via the Displacement Field , 2014, IEEE Transactions on Image Processing.

[36]  Hwang Soo Lee,et al.  Adaptive image interpolation based on local gradient features , 2004, IEEE Signal Process. Lett..

[37]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .

[38]  Michael Elad,et al.  A Statistical Prediction Model Based on Sparse Representations for Single Image Super-Resolution , 2014, IEEE Transactions on Image Processing.

[39]  Yeon Ju Lee,et al.  Nonlinear Image Upsampling Method Based on Radial Basis Function Interpolation , 2010, IEEE Transactions on Image Processing.

[40]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[41]  Lei Zhang,et al.  An edge-guided image interpolation algorithm via directional filtering and data fusion , 2006, IEEE Transactions on Image Processing.

[42]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[44]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[45]  Christine Guillemot,et al.  Single image super-resolution using sparse representations with structure constraints , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[46]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[47]  Seongjai Kim,et al.  Curvature Interpolation Method for Image Zooming , 2011, IEEE Transactions on Image Processing.

[48]  Shiguang Shan,et al.  Low-Resolution Face Recognition via Coupled Locality Preserving Mappings , 2010, IEEE Signal Processing Letters.

[49]  Shaogang Gong,et al.  Generalized Face Super-Resolution , 2008, IEEE Transactions on Image Processing.

[50]  Xinbo Gao,et al.  Single Image Super-Resolution Using Gaussian Process Regression With Dictionary-Based Sampling and Student- ${t}$ Likelihood , 2017, IEEE Transactions on Image Processing.

[51]  Michael Unser,et al.  Image interpolation and resampling , 2000 .

[52]  Yasue Mitsukura,et al.  Single image Super Resolution by no-reference image quality index optimization in PCA subspace , 2016, 2016 IEEE 12th International Colloquium on Signal Processing & Its Applications (CSPA).

[53]  Nanning Zheng,et al.  Incorporating image priors with deep convolutional neural networks for image super-resolution , 2016, Neurocomputing.

[54]  Moon Gi Kang,et al.  DCT-based regularized algorithm for high-resolution image reconstruction , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[55]  Ruimin Hu,et al.  Efficient Single Image Super-Resolution via Graph Embedding , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[56]  Thomas B. Moeslund,et al.  Super-resolution: a comprehensive survey , 2014, Machine Vision and Applications.

[57]  Yuchen Yan,et al.  Adaptive shock filter for image super-resolution and enhancement , 2016, J. Vis. Commun. Image Represent..

[58]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[59]  Klaus Diepold,et al.  Analysis Operator Learning and its Application to Image Reconstruction , 2012, IEEE Transactions on Image Processing.

[60]  Jian-Huang Lai,et al.  Face Hallucination Through KPCA , 2009, 2009 2nd International Congress on Image and Signal Processing.

[61]  Yueting Zhuang,et al.  Steerable pyramid-based face hallucination , 2005, Pattern Recognit..

[62]  Liangpei Zhang,et al.  A super-resolution reconstruction algorithm for hyperspectral images , 2012, Signal Process..

[63]  Chun Qi,et al.  Locality preserving partial least squares for neighbor embedding-based face hallucination , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[64]  Xuelong Li,et al.  Single Image Super-Resolution With Multiscale Similarity Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[65]  Christine Guillemot,et al.  Geometry-Aware Neighborhood Search for Learning Local Models for Image Superresolution , 2015, IEEE Transactions on Image Processing.

[66]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[67]  Truong Q. Nguyen,et al.  Single Image Super-Resolution Using Local Geometric Duality and Non-Local Similarity , 2016, IEEE Transactions on Image Processing.

[68]  Russell Zaretzki,et al.  Beta Process Joint Dictionary Learning for Coupled Feature Spaces with Application to Single Image Super-Resolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Karen O. Egiazarian,et al.  Single image super-resolution via BM3D sparse coding , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[71]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[72]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[73]  Sudhish N. George,et al.  Single image super resolution using neighbor embedding and statistical prediction model , 2017, Comput. Electr. Eng..

[74]  Qiong Kang,et al.  Patch-based Face Hallucination with Locality Preserving Projection , 2010, 2010 Fourth International Conference on Genetic and Evolutionary Computing.

[75]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[76]  Michael Elad,et al.  Example-Based Regularization Deployed to Super-Resolution Reconstruction of a Single Image , 2009, Comput. J..

[77]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[78]  Xiaogang Wang,et al.  Image Transformation Based on Learning Dictionaries across Image Spaces , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Shuyuan Yang,et al.  Curvelet Support Value Filters (CSVFs) for image super-resolution , 2016, Neurocomputing.

[80]  Gamal Fahmy,et al.  Exponential spline perfect reconstruction, decomposition and reconstruction with applications in compression and denoising , 2014, Signal Image Video Process..

[81]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[82]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[83]  Pascal Getreuer,et al.  Image Interpolation with Geometric Contour Stencils , 2011, Image Process. Line.

[84]  Narendra Ahuja,et al.  Super-Resolution Using Sub-Band Self-Similarity , 2014, ACCV.

[85]  Ruimin Hu,et al.  A super-resolution method for low-quality face image through RBF-PLS regression and neighbor embedding , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[86]  Wenhan Yang,et al.  Image Super-Resolution Based on Structure-Modulated Sparse Representation , 2015, IEEE Transactions on Image Processing.

[87]  Russell M. Mersereau,et al.  A Super-Resolution Framework for 3-D High-Resolution and High-Contrast Imaging Using 2-D Multislice MRI , 2009, IEEE Transactions on Medical Imaging.

[88]  Xuelong Li,et al.  Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution , 2015, IEEE Transactions on Image Processing.

[89]  Xuelong Li,et al.  SERF: A Simple, Effective, Robust, and Fast Image Super-Resolver From Cascaded Linear Regression , 2016, IEEE Transactions on Image Processing.

[90]  Xiang Ma,et al.  Sparse Support Regression for Image Super-Resolution , 2015, IEEE Photonics Journal.

[91]  Tony Jacob,et al.  Single Image Super-Resolution Using a Joint GMM Method , 2016, IEEE Transactions on Image Processing.

[92]  Yanning Zhang,et al.  Single Image Super-resolution Using Deformable Patches , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[93]  Kai-Kuang Ma,et al.  A survey on super-resolution imaging , 2011, Signal Image Video Process..

[94]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[95]  Xin Zhang,et al.  Non-local feature back-projection for image super-resolution , 2016, IET Image Process..

[96]  Liang Tang,et al.  Spatially Adaptive Block-Based Super-Resolution , 2012, IEEE Transactions on Image Processing.

[97]  Truong Q. Nguyen,et al.  Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature , 2017, IEEE Transactions on Image Processing.

[98]  Tao Li,et al.  Rotation expanded dictionary-based single image super-resolution , 2016, Neurocomputing.

[99]  Tsuhan Chen,et al.  Adaptive Markov Random Fields for Example-Based Super-resolution of Faces , 2006, EURASIP J. Adv. Signal Process..

[100]  David H. Frakes,et al.  Segment Adaptive Gradient Angle Interpolation , 2013, IEEE Transactions on Image Processing.

[101]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[102]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[103]  Sridha Sridharan,et al.  Feature-domain super-resolution framework for Gabor-based face and iris recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[104]  Aline Roumy,et al.  Single-Image Super-Resolution via Linear Mapping of Interpolated Self-Examples , 2014, IEEE Transactions on Image Processing.

[105]  Rangarajan Aravind,et al.  Face hallucination using OLPP and Kernel Ridge Regression , 2008, 2008 15th IEEE International Conference on Image Processing.

[106]  Dinesh Rajan,et al.  Unified Blind Method for Multi-Image Super-Resolution and Single/Multi-Image Blur Deconvolution , 2013, IEEE Transactions on Image Processing.

[107]  Hui Ji,et al.  Robust wavelet-based super-resolution reconstruction: theory and algorithm. , 2009, IEEE transactions on pattern analysis and machine intelligence.

[108]  Zhiliang Zhu,et al.  Fast Single Image Super-Resolution via Self-Example Learning and Sparse Representation , 2014, IEEE Transactions on Multimedia.

[109]  Shiguang Shan,et al.  Aligning Coupled Manifolds for Face Hallucination , 2009, IEEE Signal Processing Letters.

[110]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[111]  Rogério Schmidt Feris,et al.  Edge-Guided Single Depth Image Super Resolution , 2016, IEEE Transactions on Image Processing.

[112]  Hongyuan Zha,et al.  Multiscale Semilocal Interpolation With Antialiasing , 2012, IEEE Transactions on Image Processing.

[113]  Wei An,et al.  Fast multi-frame image super-resolution based on MRF , 2017, ArXiv.

[114]  Gamal Fahmy,et al.  Single image superresolution using E-SPLINE functions , 2015, 2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[115]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[116]  Joachim Weickert,et al.  Partial Differential Equations for Interpolation and Compression of Surfaces , 2008, MMCS.