Pairwise Operator Learning for Patch-Based Single-Image Super-Resolution

Motivated by the fact that image patches could be inherently represented by matrices, single-image super-resolution is treated as a problem of learning regression operators in a matrix space in this paper. The regression operators that map low-resolution image patches to high-resolution image patches are generally defined by the left and right multiplication operators. The pairwise operators are, respectively, used to extract the raw and column information of low-resolution image patches for recovering high-resolution estimations. The patch-based regression algorithm possesses three favorable properties. First, the proposed super-resolution algorithm is efficient during both training and testing, because image patches are treated as matrices. Second, the data storage requirement of the optimal pairwise operator is far less than most popular single-image super-resolution algorithms, because only two small sized matrices need to be stored. Last, the super-resolution performance is competitive with most popular single-image super-resolution algorithms, because both raw and column information of image patches is considered. Experimental results show the efficiency and effectiveness of the proposed patch-based single-image super-resolution algorithm.

[1]  Weidong Yan,et al.  Single image super resolution based on multiscale local similarity and neighbor embedding , 2016, Neurocomputing.

[2]  Hongyu Li,et al.  VSI: A Visual Saliency-Induced Index for Perceptual Image Quality Assessment , 2014, IEEE Transactions on Image Processing.

[3]  Thomas S. Huang,et al.  Robust Single Image Super-Resolution via Deep Networks With Sparse Prior , 2016, IEEE Transactions on Image Processing.

[4]  Yuan Yuan,et al.  Image Pair Analysis With Matrix-Value Operator , 2015, IEEE Transactions on Cybernetics.

[5]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[6]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[7]  Xinbo Gao,et al.  Single image super-resolution using regularization of non-local steering kernel regression , 2016, Signal Process..

[8]  Ling Shao,et al.  Blind Image Blur Estimation via Deep Learning , 2016, IEEE Transactions on Image Processing.

[9]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Xuelong Li,et al.  Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution , 2015, IEEE Transactions on Image Processing.

[11]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[12]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[13]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ling Shao,et al.  A Fast Single Image Haze Removal Algorithm Using Color Attenuation Prior , 2015, IEEE Transactions on Image Processing.

[15]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[16]  Xuelong Li,et al.  Partially Supervised Neighbor Embedding for Example-Based Image Super-Resolution , 2011, IEEE Journal of Selected Topics in Signal Processing.

[17]  Qi Wang,et al.  Example-based super-resolution via social images , 2016, Neurocomputing.

[18]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[19]  Xuelong Li,et al.  Multi-scale dictionary for single image super-resolution , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Xuelong Li,et al.  Joint Learning for Single-Image Super-Resolution via a Coupled Constraint , 2012, IEEE Transactions on Image Processing.

[21]  Hong Chen,et al.  Matrix-value regression for single-image super-resolution , 2013, 2013 International Conference on Wavelet Analysis and Pattern Recognition.

[22]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Ling Shao,et al.  Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Xuelong Li,et al.  Image Super-Resolution With Sparse Neighbor Embedding , 2012, IEEE Transactions on Image Processing.

[25]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[26]  Jie Xu,et al.  Coupled fisher discrimination dictionary learning for single image super-resolution , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Wan-Chi Siu,et al.  Single image super-resolution using Gaussian process regression , 2011, CVPR 2011.

[28]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[29]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[30]  Neeraj Kumar,et al.  Fast Learning-Based Single Image Super-Resolution , 2016, IEEE Transactions on Multimedia.

[31]  Xuelong Li,et al.  Single image super resolution with high resolution dictionary , 2011, 2011 18th IEEE International Conference on Image Processing.

[32]  Shiguang Shan,et al.  Locality preserving constraints for super-resolution with neighbor embedding , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[33]  Hong Chang,et al.  Super-resolution through neighbor embedding , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[34]  Ling Shao,et al.  Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Xuelong Li,et al.  A Unified Learning Framework for Single Image Super-Resolution , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[37]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[38]  Zhou Wang,et al.  Image Quality Assessment: From Error Measurement to Structural Similarity , 2004 .

[39]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[40]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[41]  A. Basarab,et al.  Fast Single Image Super-resolution using a New Analytical Solution for l2-l2 Problems. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[42]  Xinbo Gao,et al.  Single-Image Super-Resolution Using Active-Sampling Gaussian Process Regression , 2016, IEEE Transactions on Image Processing.

[43]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  Jean-Yves Tourneret,et al.  Fast Single Image Super-Resolution Using a New Analytical Solution for $\ell _{2}$ – $\ell _{2}$ Problems , 2016, IEEE Transactions on Image Processing.

[45]  Jianzhong Wang,et al.  Locally Linear Embedding , 2021, Unsupervised Learning Approaches for Dimensionality Reduction and Data Visualization.

[46]  KumarNeeraj,et al.  Fast Learning-Based Single Image Super-Resolution , 2016 .

[47]  Xuelong Li,et al.  Greedy regression in sparse coding space for single-image super-resolution , 2013, J. Vis. Commun. Image Represent..

[48]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[49]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[50]  Xuelong Li,et al.  Geometry constrained sparse coding for single image super-resolution , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[52]  Truong Q. Nguyen,et al.  Image Superresolution Using Support Vector Regression , 2007, IEEE Transactions on Image Processing.