Single image super-resolution via low-rank tensor representation and hierarchical dictionary learning

Super-resolution (SR) has been widely studied due to its importance in real applications and scenarios. In this paper, we focus on generating an SR image from a single low-resolution (LR) input image by employing the multi-resolution structures of an input image. By taking the LR image and its downsampled resolution (DR) and upsampled resolution (UR) versions as inputs, we propose a hierarchical dictionary learning approach to learn the latent UR-LR dictionary pair by preserving the internal structure coherence with the LR-DR dictionary pair. Note that an imposed restriction involved in this process is that the pairwise resolution images are jointly trained to obtain more compact patterns of image patches. In particular, to better explore the underlying structures of tensor data spanned by image patches, we propose a low-rank tensor approximation (LRTA) algorithm based on nuclear-norm regularization to embed input image patches into a low-dimensional space. Experimental results from publicly used images show that our proposed method achieves performance comparable with that of other state-of-the-art SR algorithms, even without using any external training databases.

[1]  Xiao Liu,et al.  Semi-supervised Coupled Dictionary Learning for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Peyman Milanfar,et al.  A computationally efficient superresolution image reconstruction algorithm , 2001, IEEE Trans. Image Process..

[3]  Xiaoyan Sun,et al.  TenSR: Multi-dimensional Tensor Sparse Representation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Mohan S. Kankanhalli,et al.  MMALFM , 2018, ACM Trans. Inf. Syst..

[6]  Rama Chellappa,et al.  Cross-View Action Recognition via Transferable Dictionary Learning , 2016, IEEE Transactions on Image Processing.

[7]  Naoto Yokoya,et al.  Hyperspectral Super-Resolution of Locally Low Rank Images From Complementary Multisource Data , 2014, IEEE Transactions on Image Processing.

[8]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[9]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[10]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[11]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[12]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Wei Liu,et al.  Tensor Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Tensors via Convex Optimization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Jiejie Zhu,et al.  Context-constrained hallucination for image super-resolution , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Xuelong Li,et al.  Multi-scale dictionary for single image super-resolution , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[17]  Chih-Yuan Yang,et al.  Single-Image Super-Resolution: A Benchmark , 2014, ECCV.

[18]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Meng Wang,et al.  Towards Micro-video Understanding by Joint Sequential-Sparse Modeling , 2017, ACM Multimedia.

[20]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[24]  Yasuyuki Matsushita,et al.  High-resolution hyperspectral imaging via matrix factorization , 2011, CVPR 2011.

[25]  Rebecca Willett,et al.  Compressive coded aperture superresolution image reconstruction , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Aline Roumy,et al.  Low-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding , 2012, BMVC.

[28]  Zhe L. Lin,et al.  Fast Image Super-Resolution Based on In-Place Example Regression , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Jialie Shen,et al.  Forbidden City Great Wall Old SummerPalace Temple of Heaven Tiananmen Square Avenue of Stars Disneyland Resort Peninsular Hotel Tian Tan Budda Victoria Harbour Big Ben Buckingham Palace , 2016 .

[30]  Xuelong Li,et al.  Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution , 2015, IEEE Transactions on Image Processing.

[31]  Hayit Greenspan,et al.  Super-Resolution in Medical Imaging , 2009, Comput. J..

[32]  Tat-Seng Chua,et al.  Learning from Multiple Social Networks , 2016, Synthesis Lectures on Information Concepts, Retrieval, and Services.

[33]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[34]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[35]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[36]  Feiping Nie,et al.  Low-Rank Tensor Completion with Spatio-Temporal Consistency , 2014, AAAI.

[37]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[38]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[39]  Guangming Shi,et al.  Low-Rank Tensor Approximation with Laplacian Scale Mixture Modeling for Multiframe Image Denoising , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[40]  Chun Qi,et al.  Low-Rank Neighbor Embedding for Single Image Super-Resolution , 2014, IEEE Signal Processing Letters.

[41]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[42]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[43]  Narendra Ahuja,et al.  Super-resolving Noisy Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Xiaogang Wang,et al.  Image Transformation Based on Learning Dictionaries across Image Spaces , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Michal Irani,et al.  Internal statistics of a single natural image , 2011, CVPR 2011.

[46]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[47]  Xiao-Yuan Jing,et al.  Super-resolution Person re-identification with semi-coupled low-rank discriminant dictionary learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  D. Shen,et al.  LRTV: MR Image Super-Resolution With Low-Rank and Total Variation Regularizations , 2015, IEEE Transactions on Medical Imaging.

[49]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[50]  Guoqiang Zhong,et al.  Low-Rank Tensor Learning with Discriminant Analysis for Action Classification and Image Recovery , 2014, AAAI.

[51]  Baowen Xu,et al.  Super-resolution Person re-identification with semi-coupled low-rank discriminant dictionary learning , 2015, CVPR.

[52]  Xinyuan Zhang,et al.  Nonlocal Low-Rank Tensor Factor Analysis for Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.