Single image super resolution via wavelet transform fusion and SRFeat network

Image super resolution is a vital research topic in the field of computer vision. It aims to reconstruct high resolution images from low resolution images. Although the conventional image super resolution methods have achieved good performance and effect, there are still have some issues, e.g., the high-frequency details information is insufficient, and the reconstruction process will bring additional noise, and most basic interpolation techniques produce blurry results. To settle the problems mentioned above, we consider combining the deep learning method with the frequency domain fusion method. In this paper, a novel single image super resolution method based on SRFeat network and wavelet fusion is proposed. First, the training image is taken as the input of the backbone SRFeat network, then the generative adversarial network training is carried out. Then, the up-sampling is utilized to obtain the coarse super resolved image. Finally, the output image after the network training is combined with the up-sampling image of the low-resolution image by Wavelet fusion to obtain the final result. Without increasing the depth of the network and the redundant parameters, the proposed method can achieve better reconstruct result. The experimental results show that the proposed method can not only reduce the probability of image distortion, but recover the global information of the reconstructed image and remove the noise brought by the reconstruction process. The PSNR value of the proposed method is improved 0.3 dB, and the SSIM is improved 0.02.

[1]  Xuelong Li,et al.  Single Image Super-Resolution With Non-Local Means and Steering Kernel Regression , 2012, IEEE Transactions on Image Processing.

[2]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[4]  Michal Irani,et al.  Super resolution from image sequences , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[5]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Robert L. Stevenson,et al.  Motion-compensated scan conversion of interlaced video sequences , 1996, Electronic Imaging.

[8]  Huimin Lu,et al.  Ternary Adversarial Networks With Self-Supervision for Zero-Shot Cross-Modal Retrieval , 2020, IEEE Transactions on Cybernetics.

[9]  Thomas G. Dietterich.  Adaptive computation and machine learning , 1998 .

[10]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Seungyong Lee,et al.  SRFeat: Single Image Super-Resolution with Feature Discrimination , 2018, ECCV.

[12]  Henry Stark,et al.  Projection-based image restoration , 1992 .

[13]  Xi Chen,et al.  Image denoising via deep network based on edge enhancement , 2018, Journal of Ambient Intelligence and Humanized Computing.

[14]  Huimin Lu,et al.  CONet: A Cognitive Ocean Network , 2019, IEEE Wireless Communications.

[15]  Fei Zhou,et al.  Interpolation-Based Image Super-Resolution Using Multisurface Fitting , 2012, IEEE Transactions on Image Processing.

[16]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Heng Tao Shen,et al.  Cross-Modal Attention With Semantic Consistence for Image–Text Matching , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jinde Cao,et al.  Pulse coupled neural network based MRI image enhancement using classical visual receptive field for smarter mobile healthcare , 2018, J. Ambient Intell. Humaniz. Comput..