Brief Survey of Single Image Super-Resolution Reconstruction Based on Deep Learning Approaches

With the presentation of super-resolution convolutional neural network, deep learning approach was applied to image super-resolution reconstruction for the first time. By using convolutional neural network, the deep learning approaches can directly learn the mapping relationship between the low-resolution image and high-resolution image, and have achieved better reconstruction effects than the traditional image super-resolution reconstruction methods. Subsequently, a series of improved deep learning approaches have been proposed, and the reconstruction effects have been improved continuously. This paper systematically summa rizes the image super-resolution reconstruction approaches based on deep learning, analyzes the characteristics of different models, and compares the main deep learning models based on the experiments. Furthermore, based on deep learning model, the future research directions of the image super-resolution reconstruction methods based on deep learning models are reasonably predicted.

[1]  Nam Ik Cho,et al.  Age and gender estimation using deep residual learning network , 2018, 2018 International Workshop on Advanced Image Technology (IWAIT).

[2]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Karthik Ramani,et al.  SurfNet: Generating 3D Shape Surfaces Using Deep Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Han Jiang,et al.  Deep residual networks for hyperspectral image classification , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[5]  Liang Xiao,et al.  PAN-Sharpening via residual deep learning , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[6]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Bodo Rosenhahn,et al.  PSyCo: Manifold Span Reduction for Super Resolution , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[9]  Horst Bischof,et al.  Fast and accurate image upscaling with super-resolution forests , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Chao Dong,et al.  Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Jing Yang,et al.  To learn image super-resolution, use a GAN to learn how to do image degradation first , 2018, ECCV.

[12]  Daniel Rueckert,et al.  Cardiac Image Super-Resolution with Global Correspondence Using Multi-Atlas PatchMatch , 2013, MICCAI.

[13]  Graham W. Taylor,et al.  Adaptive deconvolutional networks for mid and high level feature learning , 2011, 2011 International Conference on Computer Vision.

[14]  Hyung Soon Kim,et al.  Multi-resolution DenseNet based acoustic models for reverberant speech recognition , 2018 .

[15]  Wang Wei,et al.  Image Object Recognition via Deep Feature-Based Adaptive Joint Sparse Representation , 2019 .

[16]  Jordi Salvador,et al.  Naive Bayes Super-Resolution Forest , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Yanhong Luo,et al.  An Advanced Deep Residual Dense Network (DRDN) Approach for Image Super-Resolution , 2019, Int. J. Comput. Intell. Syst..

[18]  Pong C. Yuen,et al.  Very low resolution face recognition problem , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[19]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[24]  Fei Yin,et al.  Handwritten Chinese character recognition with spatial transformer and deep residual networks , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[25]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Wei Wang,et al.  A Novel Image Classification Approach via Dense-MobileNet Models , 2020, Mob. Inf. Syst..

[27]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[28]  Hongyu Wang,et al.  End-to-End Image Super-Resolution via Deep and Shallow Convolutional Networks , 2016, IEEE Access.

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[31]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Weizheng Wang,et al.  Development of convolutional neural network and its application in image classification: a survey , 2019, Optical Engineering.

[33]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[34]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[35]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Zhe Gan,et al.  AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[39]  Kwang In Kim,et al.  Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).