Convolutional Sparse Coding Using Wavelets for Single Image Super-Resolution

In this paper, we propose the convolutional sparse coding based model in the wavelet domain for the task of single image super-resolution (SISR). The conventional sparse coding based approaches work on overlapping image patches and use the dictionary atoms to sparse code an image patch. Further, at the final stage, an overlap-add mechanism is used to get the final high-resolution image estimate. However, these algorithms fail to take into account the consistency present in the overlapping patches which limits their performance. We propose the use of wavelet integrated convolutional sparse coding approach where instead of dictionary atoms we utilize the convolution summations between the learned filters and their mappings for sparse representation based SISR. The use of wavelets is proposed owing to their unique directional and compact features. A pair of filters are learned along with a mapping function for each wavelet sub-band to exploit the consistency among patches. The proposed wavelet integrated convolutional sparse coding model helps capture useful contextual information. The proposed model is evaluated on publicly available datasets for different scale-up parameters. To show the efficacy of the proposed model we compare it with recent state-of-the-art algorithms. The visual results along with the quantitative ones indicate that the proposed model performs well for the tasks of super-resolution.

[1]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[3]  Brendt Wohlberg,et al.  Efficient convolutional sparse coding , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[5]  Matthias Zwicker,et al.  Deep Mean-Shift Priors for Image Restoration , 2017, NIPS.

[6]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[7]  Reinhard Klette,et al.  Coupled dictionary learning in wavelet domain for Single-Image Super-Resolution , 2018, Signal Image Video Process..

[8]  Murat Ekinci,et al.  Single image super resolution based on sparse representation using discrete wavelet transform , 2017, Multimedia Tools and Applications.

[9]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[10]  Wangmeng Zuo,et al.  Learning a Single Convolutional Super-Resolution Network for Multiple Degradations , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Qiang Zhang,et al.  Hyperspectral Image Denoising Employing a Spatial–Spectral Deep Residual Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[14]  Liang Lin,et al.  Multi-level Wavelet-CNN for Image Restoration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Chengdong Wu,et al.  Learning Weighted Forest and Similar Structure for Image Super Resolution , 2019, Applied Sciences.

[16]  Y. H. Song,et al.  Microarray image de-noising using stationary wavelet transform , 2003, 4th International IEEE EMBS Special Topic Conference on Information Technology Applications in Biomedicine, 2003..

[17]  Jong Chul Ye,et al.  Beyond Deep Residual Learning for Image Restoration: Persistent Homology-Guided Manifold Simplification , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Lei Zhang,et al.  Convolutional Sparse Coding for Image Super-Resolution , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Chao Zeng,et al.  Missing Data Reconstruction in Remote Sensing Image With a Unified Spatial–Temporal–Spectral Deep Convolutional Neural Network , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Mahmoud Nazzal,et al.  Wavelet domain dictionary learning-based single image superresolution , 2015, Signal Image Video Process..

[23]  Vishal Monga,et al.  Deep Wavelet Prediction for Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[24]  Neeraj Kumar,et al.  Convolutional neural networks for wavelet domain super resolution , 2017, Pattern Recognit. Lett..

[25]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[27]  Gui Yun Tian,et al.  Wavelet domain based directional dictionaries for single image super-resolution , 2017, 2017 IEEE International Conference on Imaging Systems and Techniques (IST).

[28]  Xinbo Gao,et al.  Fast and Accurate Single Image Super-Resolution via Information Distillation Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Anders P. Eriksson,et al.  Fast Convolutional Sparse Coding , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  J. A. Parker,et al.  Comparison of Interpolating Methods for Image Resampling , 1983, IEEE Transactions on Medical Imaging.

[31]  Lei Zhang,et al.  Nonlocally Centralized Sparse Representation for Image Restoration , 2013, IEEE Transactions on Image Processing.

[32]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.