A New Super Resolution Framework Based on Multi-Task Learning for Remote Sensing Images

Super-resolution (SR) algorithms based on deep learning have dominated in various tasks, including medical imaging, street view surveillance and face recognition. In the remote sensing field, most of the current SR methods utilize the low-resolution (LR) images that directly bicubic downsampled the high-resolution (HR) images as not only train set but also test set, thus achieving high PSNR/SSIM scores but showing performance drop in application because the degradation model in remote sensing images is subjected to Gaussian blur with unknown parameters. Inspired by multi-task learning strategy, we propose a multiple-blur-kernel super-resolution framework (MSF), in which a multiple-blur-kernel learning module (MLM) optimizes the parameters of the network transferable and sensitive for SR procedures with different blur kernels. Besides, to simultaneously exploit the prior of the large-scale remote sensing images and recurrent information in a single test image, a class-feature capture module (CCM) and an unsupervised learning module (ULM) are leveraged in our framework. Extensive experiments show that our framework outperforms the current state-of-the-art SR algorithms in remotely sensed imagery SR with unknown Gaussian blur kernel.

[1]  Siyuan Liu,et al.  Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Michal Irani,et al.  Nonparametric Blind Super-resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[3]  Wei Wu,et al.  Feedback Network for Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Gang Wan,et al.  Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark , 2020, ISPRS Journal of Photogrammetry and Remote Sensing.

[5]  Lingli Zhao,et al.  A residual convolutional neural network for polarimetric SAR image super-resolution , 2020, ISPRS Journal of Photogrammetry and Remote Sensing.

[6]  Christian Ledig,et al.  Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[12]  Konrad Schindler,et al.  Super-Resolution of Sentinel-2 Images: Learning a Globally Applicable Deep Neural Network , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[13]  Nick Barnes,et al.  Densely Residual Laplacian Super-Resolution , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Jian Yang,et al.  Image Super-Resolution via Deep Recursive Residual Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[16]  Shu-Tao Xia,et al.  Second-Order Attention Network for Single Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.