Classification of priors and regularization techniques appurtenant to single image super-resolution

Image super-resolution (SR) is the process of restoration of high-resolution (HR) image from its degraded images/image. Exigency of high-quality images in different technical fields has led it to be one of the prominent research domains in the area of digital image processing. In SR process, image reconstruction from single low-resolution (LR) image is more onerous process than obtaining it from multi-LR images. Performance of single image SR (SISR) processes is increased by utilizing different image priors in form of regularization. Use of regularization in mathematical equation helps in better visualization and reconstruction of original HR image. Its implementation requires maneuver and is a very important part of the image reconstruction process. In this paper, an attempt has been made to classify the existing techniques based on the priors used. Types of image priors and regularization used so far in the field of SISR have been discussed in detailed, comprising the recent advancements in this area. It also consists of the problems and limitations of the existing image priors and regularizations.

[1]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[2]  Jie Li,et al.  Image super-resolution: The techniques, applications, and future , 2016, Signal Process..

[3]  Guangming Shi,et al.  Image Restoration via Simultaneous Sparse Coding: Where Structured Sparsity Meets Gaussian Scale Mixture , 2015, International Journal of Computer Vision.

[4]  Jinming Li.  Sparse representation based single image super-resolution with low-rank constraint and nonlocal self-similarity , 2017, Multimedia Tools and Applications.

[5]  Aline Roumy,et al.  Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Madad Ali Shah,et al.  Single image super-resolution by directionally structured coupled dictionary learning , 2016, EURASIP J. Image Video Process..

[7]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA 2013, Kaohsiung, Taiwan, October 29 - November 1, 2013 , 2013, APSIPA.

[9]  Subhasis Chaudhuri,et al.  Single-Frame Image Super-resolution through Contourlet Learning , 2006, EURASIP J. Adv. Signal Process..

[10]  Moawad I. Dessouky,et al.  Regularized super-resolution reconstruction of images using wavelet fusion , 2005 .

[11]  Narendra Ahuja,et al.  Sub-band Energy Constraints for Self-Similarity Based Super-resolution , 2014, 2014 22nd International Conference on Pattern Recognition.

[12]  Jie Jiang,et al.  Super-resolution reconstruction based on structure tensor's eigenvalue and classification dictionary , 2016, 2016 International Conference on Audio, Language and Image Processing (ICALIP).

[13]  Peyman Milanfar,et al.  Robust Kernel Regression for Restoration and Reconstruction of Images from Sparse Noisy Data , 2006, 2006 International Conference on Image Processing.

[14]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[15]  Jae-Seok Choi,et al.  Single Image Super-Resolution Using Global Regression Based on Multiple Local Linear Mappings , 2017, IEEE Transactions on Image Processing.

[16]  Vivek Bannore,et al.  Regularization for Super-Resolution Image Reconstruction , 2006, KES.

[17]  Chaomin Shen,et al.  Single Image Super-Resolution Based on Nonlocal Sparse and Low-Rank Regularization , 2016, PRICAI.

[18]  Xuelong Li,et al.  Joint Learning for Single-Image Super-Resolution via a Coupled Constraint , 2012, IEEE Transactions on Image Processing.

[19]  Pascal Getreuer,et al.  Image Interpolation with Geometric Contour Stencils , 2011, Image Process. Line.

[20]  Aggelos K. Katsaggelos,et al.  Bayesian combination of sparse and non-sparse priors in image super resolution , 2013, Digit. Signal Process..

[21]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Michael Elad,et al.  Generalizing the Nonlocal-Means to Super-Resolution Reconstruction , 2009, IEEE Transactions on Image Processing.

[23]  Guangming Shi,et al.  Image Super-Resolution With Parametric Sparse Model Learning , 2018, IEEE Transactions on Image Processing.

[24]  Ahmad Reza Naghsh-Nilchi,et al.  High-order Markov random field for single depth image super-resolution , 2017, IET Comput. Vis..

[25]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Dinesh Rajan,et al.  Unified Blind Method for Multi-Image Super-Resolution and Single/Multi-Image Blur Deconvolution , 2013, IEEE Transactions on Image Processing.

[27]  Xuelong Li,et al.  Greedy regression in sparse coding space for single-image super-resolution , 2013, J. Vis. Commun. Image Represent..

[28]  Ling Shao,et al.  Pairwise Operator Learning for Patch-Based Single-Image Super-Resolution , 2017, IEEE Transactions on Image Processing.

[29]  Yuan Xie,et al.  Single image super-resolution using combined total variation regularization by split Bregman Iteration , 2014, Neurocomputing.

[30]  Weiguo Gong,et al.  Combining edge difference with nonlocal self-similarity constraints for single image super-resolution , 2017, Neurocomputing.

[31]  Wan-Chi Siu,et al.  Robust Soft-Decision Interpolation Using Weighted Least Squares , 2012, IEEE Transactions on Image Processing.

[32]  Amisha J. Shah,et al.  A Technique to Preserve Edge Information in Single Image Super Resolution , 2016 .

[33]  Jing Hu,et al.  Single-image superresolution based on local regression and nonlocal self-similarity , 2014, J. Electronic Imaging.

[34]  Yanan Liu,et al.  Super-resolution image reconstruction using fractional-order total variation and adaptive regularization parameters , 2018, The Visual Computer.

[35]  Prakash P. Gajjar,et al.  Zoom based image super-resolution using DCT with LBP as characteristic model , 2018 .

[36]  Tianming Zhan,et al.  A Novel 3D Anisotropic Total Variation Regularized Low Rank Method for Hyperspectral Image Mixed Denoising , 2018, ISPRS Int. J. Geo Inf..

[37]  Stephen J. Roberts,et al.  A Sampled Texture Prior for Image Super-Resolution , 2003, NIPS.

[38]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[39]  Kaamran Raahemifar,et al.  A performance comparison among different super-resolution techniques , 2016, Comput. Electr. Eng..

[40]  Pier Luigi Dragotti,et al.  FRESH—FRI-Based Single-Image Super-Resolution Algorithm , 2016, IEEE Transactions on Image Processing.

[41]  Amisha J. Shah,et al.  Single Image Super-Resolution via Non Sub-sample Contourlet Transform based Learning and a Gabor Prior , 2013 .

[42]  Zheng Liu,et al.  Single-image super-resolution based on Markov random field and contourlet transform , 2011, J. Electronic Imaging.

[43]  Horst Bischof,et al.  Variational Depth Superresolution Using Example-Based Edge Representations , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[44]  Hongyuan Zha,et al.  Multiscale Semilocal Interpolation With Antialiasing , 2012, IEEE Transactions on Image Processing.

[45]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[46]  Wen Gao,et al.  Depth Super-Resolution via Joint Color-Guided Internal and External Regularizations , 2019, IEEE Transactions on Image Processing.

[47]  Nirmal K. Bose,et al.  Simultaneous noise filtering and super-resolution with second-generation wavelets , 2005, IEEE Signal Processing Letters.

[48]  Karen O. Egiazarian,et al.  Single Image Super-Resolution Based on Wiener Filter in Similarity Domain , 2017, IEEE Transactions on Image Processing.

[49]  Michael Elad,et al.  Advances and challenges in super‐resolution , 2004, Int. J. Imaging Syst. Technol..

[50]  Bruno A. Olshausen,et al.  Group Sparse Coding with a Laplacian Scale Mixture Prior , 2010, NIPS.

[51]  Wen Gao,et al.  Image Interpolation Via Regularized Local Linear Regression , 2011, IEEE Transactions on Image Processing.

[52]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[53]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[54]  Yongsheng Zuo,et al.  Spherically contoured exponential scale mixture prior based nonlocal image restoration with ADMM framework , 2018, J. Vis. Commun. Image Represent..

[55]  Kai-Kuang Ma,et al.  A survey on super-resolution imaging , 2011, Signal Image Video Process..

[56]  Yang Wang,et al.  Single image super-resolution reconstruction based on genetic algorithm and regularization prior model , 2016, Inf. Sci..

[57]  Qionghai Dai,et al.  Single depth image super-resolution and denoising based on sparse graphs via structure tensor , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[58]  M. Wilscy,et al.  Single image super-resolution based on compressive sensing and improved TV minimization sparse recovery , 2017 .

[59]  Thomas B. Moeslund,et al.  Super-resolution: a comprehensive survey , 2014, Machine Vision and Applications.

[60]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[61]  Thomas S. Huang,et al.  Coupled Dictionary Training for Image Super-Resolution , 2012, IEEE Transactions on Image Processing.

[62]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[63]  Truong Q. Nguyen,et al.  Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature , 2017, IEEE Transactions on Image Processing.

[64]  Thomas S. Huang,et al.  Robust Single Image Super-Resolution via Deep Networks With Sparse Prior , 2016, IEEE Transactions on Image Processing.

[65]  Tomer Michaeli,et al.  Visualizing Image Priors , 2016, ECCV.

[66]  Xuelong Li,et al.  Geometry constrained sparse coding for single image super-resolution , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[67]  Yücel Altunbasak,et al.  Eigenface-based super-resolution for face recognition , 2002, Proceedings. International Conference on Image Processing.

[68]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Moon Gi Kang,et al.  Super-resolution image reconstruction: a technical overview , 2003, IEEE Signal Process. Mag..

[70]  Christine Guillemot,et al.  Single image super-resolution using sparse representations with structure constraints , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[71]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[72]  Changick Kim,et al.  Discrete feature transform for low-complexity single-image super-resolution , 2016, 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[73]  Jin Hyung Kim,et al.  Efficient Learning of Image Super-Resolution and Compression Artifact Removal with Semi-Local Gaussian Processes , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Umesh Ghanekar,et al.  A compendious study of super-resolution techniques by single image , 2018 .

[75]  Joonki Paik,et al.  Single image super-resolution using locally adaptive multiple linear regression. , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[76]  Truong Q. Nguyen,et al.  Single Image Super-Resolution Using Local Geometric Duality and Non-Local Similarity , 2016, IEEE Transactions on Image Processing.

[77]  Kwang In Kim,et al.  Example-Based Learning for Single-Image Super-Resolution , 2008, DAGM-Symposium.

[78]  Sudhish N. George,et al.  Robust single image super resolution using neighbor embedding and fusion in wavelet domain , 2017, Comput. Electr. Eng..

[79]  Truong Q. Nguyen,et al.  Image Superresolution Using Support Vector Regression , 2007, IEEE Transactions on Image Processing.

[80]  Guo Cao,et al.  Single image super-resolution via adaptive sparse representation and low-rank constraint , 2018, J. Vis. Commun. Image Represent..

[81]  Bryan C. Russell,et al.  Exploiting the sparse derivative prior for super-resolution , 2003 .

[82]  Mehran Ebrahimi,et al.  Solving the Inverse Problem of Image Zooming Using "Self-Examples" , 2007, ICIAR.

[83]  Stanley Osher,et al.  Image Super-Resolution by TV-Regularization and Bregman Iteration , 2008, J. Sci. Comput..

[84]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[85]  Emmanuel J. Candès,et al.  Super-resolution via Transform-Invariant Group-Sparse Regularization , 2013, 2013 IEEE International Conference on Computer Vision.

[86]  Russell Zaretzki,et al.  Beta Process Joint Dictionary Learning for Coupled Feature Spaces with Application to Single Image Super-Resolution , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[87]  Gamal Fahmy,et al.  Single image superresolution using E-SPLINE functions , 2015, 2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[88]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[89]  Barak A. Pearlmutter,et al.  Survey of sparse and non‐sparse methods in source separation , 2005, Int. J. Imaging Syst. Technol..

[90]  Joachim Weickert,et al.  Partial Differential Equations for Interpolation and Compression of Surfaces , 2008, MMCS.

[91]  Tingting Wu,et al.  Image-zooming technique based on Bregmanized nonlocal total variation regularization , 2011 .

[92]  Isao Echizen,et al.  Limitations of super resolution image reconstruction and how to overcome them for a single image , 2013, 2013 International Conference on Signal Processing and Multimedia Applications (SIGMAP).

[93]  Hajime Nobuhara,et al.  First-order derivative-based super-resolution , 2017, Signal Image Video Process..

[94]  Luc Van Gool,et al.  Jointly Optimized Regressors for Image Super‐resolution , 2015, Comput. Graph. Forum.

[95]  Truong Q. Nguyen,et al.  Markov Random Field Model-Based Edge-Directed Image Interpolation , 2007, IEEE Transactions on Image Processing.

[96]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[97]  Georgios C. Anagnostopoulos,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2003, Lecture Notes in Computer Science.

[98]  Karen O. Egiazarian,et al.  Single image super-resolution via BM3D sparse coding , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[99]  Lei Zhang,et al.  Sparse Representation Based Image Interpolation With Nonlocal Autoregressive Modeling , 2013, IEEE Transactions on Image Processing.

[100]  Xiaolei Lu,et al.  ADMM for image restoration based on nonlocal simultaneous sparse Bayesian coding , 2019, Signal Process. Image Commun..

[101]  Vincy Salam,et al.  A New Approach for Super resolution by Using Web Images and FFT Based Image Registration , 2014, ArXiv.

[102]  Thomas S. Huang,et al.  Non-Local Kernel Regression for Image and Video Restoration , 2010, ECCV.

[103]  Moon Gi Kang,et al.  DCT-based regularized algorithm for high-resolution image reconstruction , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[104]  Lei Zhang,et al.  Centralized sparse representation for image restoration , 2011, 2011 International Conference on Computer Vision.

[105]  Tomio Goto,et al.  Super-resolution System for 4K-HDTV , 2014, 2014 22nd International Conference on Pattern Recognition.

[106]  Guangming Shi,et al.  Learning Parametric Distributions for Image Super-Resolution: Where Patch Matching Meets Sparse Coding , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[107]  Dacheng Tao,et al.  Single Image Superresolution via Directional Group Sparsity and Directional Features , 2015, IEEE Transactions on Image Processing.

[108]  Chun Qi,et al.  Low-rank sparse representation for single image super-resolution via self-similarity learning , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[109]  LiJie,et al.  Image super-resolution , 2016 .

[110]  Rynson W. H. Lau,et al.  Efficient image super-resolution integration , 2018, The Visual Computer.

[111]  Xuelong Li,et al.  Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution , 2015, IEEE Transactions on Image Processing.

[112]  Jie Liu,et al.  Sparse representation and adaptive mixed samples regression for single image super-resolution , 2018, Signal Process. Image Commun..

[113]  Zhe L. Lin,et al.  Fast Image Super-Resolution Based on In-Place Example Regression , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[114]  Quan Pan,et al.  Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[115]  Aggelos K. Katsaggelos,et al.  A non-stationary image prior combination in super-resolution , 2014, Digit. Signal Process..