Image super-resolution reconstruction based on sparse representation and deep learning

Abstract Super-resolution reconstruction technology has important scientific significance and application value in the field of image processing by performing image restoration processing on one or more low-resolution images to improve image spatial resolution. Based on the SCSR algorithm and VDSR network, in order to further improve the image reconstruction quality, an image super-resolution reconstruction algorithm combined with multi-residual network and multi-feature SCSR(MRMFSCSR) is proposed. Firstly, at the sparse reconstruction stage, according to the characteristics of image blocks, our algorithm extracts the contour features of non-flat blocks by NSCT transform, extracts the texture features of flat blocks by Gabor transform, then obtains the reconstructed high-resolution (HR) images by using sparse models. Secondly, according to improve the VDSR deep network and introduce the feature fusion idea, the multi-residual network structure (MR) is designed. The reconstructed HR image obtained by the sparse reconstruction stage is used as the input of the MR network structure to optimize the high-frequency detail residual information. Finally, we can obtain a higher quality super-resolution image compared with the SCSR algorithm and the VDSR algorithm.

[1]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Mei Han,et al.  Soft Edge Smoothness Prior for Alpha Channel Super Resolution , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Hsieh Hou,et al.  Cubic splines for image interpolation and digital filtering , 1978 .

[5]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Christopher M. Bishop,et al.  Bayesian Image Super-Resolution , 2002, NIPS.

[7]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[8]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[9]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[11]  Takeo Kanade,et al.  Limits on Super-Resolution and How to Break Them , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Thomas S. Huang,et al.  Deep Networks for Image Super-Resolution with Sparse Prior , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Luc Van Gool,et al.  A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution , 2014, ACCV.

[16]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[17]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.