Nonlocal Similarity Modeling and Deep CNN Gradient Prior for Super Resolution

This letter presents a novel super-resolution (SR) method via nonlocal similarity modeling and deep convolutional neural network (CNN) gradient prior (GP). Specifically, on the one hand, the group similarity reliability (GSR) strategy is proposed for improving the adaptive high-dimensional nonlocal total variation (AHNLTV) model [statistical prior, GSR-based AHNLTV (GA)], which captures the structures of the underlying high-resolution (HR) image via the image itself. On the other hand, the GP is learned by using the deep CNN (learned prior), which predicts the gradients from external images. Finally, the GA–GP approach is proposed by incorporating the two complementary priors. The results show that GA–GP achieves better performance than other state-of-the-art SR methods.

[1]  Zixiang Xiong,et al.  DLML: Deep linear mappings learning for face super-resolution with nonlocal-patch , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[2]  Junjun Jiang,et al.  Single Image Super-Resolution via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means , 2017, IEEE Transactions on Multimedia.

[3]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  Ruimin Hu,et al.  Facial Image Hallucination Through Coupled-Layer Neighbor Embedding , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[7]  Li Yi,et al.  Image Super-Resolution Via Analysis Sparse Prior , 2013, IEEE Signal Processing Letters.

[8]  Jan Kautz,et al.  Loss Functions for Image Restoration With Neural Networks , 2017, IEEE Transactions on Computational Imaging.

[9]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[10]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[11]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[12]  Truong Q. Nguyen,et al.  Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature , 2017, IEEE Transactions on Image Processing.

[13]  Shiguang Shan,et al.  Deep Network Cascade for Image Super-resolution , 2014, ECCV.

[14]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[17]  Weiguo Gong,et al.  Non-blind image deblurring method by local and nonlocal total variation models , 2014, Signal Process..

[18]  Dong Xu,et al.  Example-Based Super-Resolution With Soft Information and Decision , 2013, IEEE Transactions on Multimedia.

[19]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[21]  Truong Q. Nguyen,et al.  Single Image Superresolution Based on Gradient Profile Sharpness , 2015, IEEE Transactions on Image Processing.

[22]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Jean Ponce,et al.  Learning a convolutional neural network for non-uniform motion blur removal , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Michael Elad,et al.  Single Image Interpolation Via Adaptive Nonlocal Sparsity-Based Modeling , 2014, IEEE Transactions on Image Processing.

[28]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[29]  Michael Unser,et al.  CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[30]  L. Cohen,et al.  Non-local Regularization of Inverse Problems , 2008, ECCV.

[31]  Xinfeng Zhang,et al.  Retrieval Compensated Group Structured Sparsity for Image Super-Resolution , 2017, IEEE Transactions on Multimedia.

[32]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[33]  Truong Q. Nguyen,et al.  Single Image Super-Resolution Using Local Geometric Duality and Non-Local Similarity , 2016, IEEE Transactions on Image Processing.

[34]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[35]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[36]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[37]  Michael Elad,et al.  Multi-Scale Patch-Based Image Restoration , 2016, IEEE Transactions on Image Processing.

[38]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[39]  Horst Bischof,et al.  Conditioned Regression Models for Non-blind Single Image Super-Resolution , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).