GAN-Based Multi-level Mapping Network for Satellite Imagery Super-Resolution

Although many deep-learning-based image super-resolution (SR) methods have been proposed, most of them assume that all hierarchical features share the unified mapping equations. They ignore the differences between mapping equations at different feature levels, and create an average effect of mapping prediction, thus poorly building the mapping relations between low resolution (LR) and high resolution (HR) spaces. In this paper, we propose a multi-level mapping framework along with the adversarial learning strategy, namely MMGAN, for satellite imageries SR reconstruction. We also construct a feature extraction and tuning block (FETB) for fine feature expression. In particular, a novel two-dimension dense unit (DU) and a mapping attention unit (MAU) are constructed for building multi-level mappings in different stages. With our strategies, an HR image is reconstructed directly from the input image using multi-level mappings. Extensive experiments on Kaggle Open Source Dataset and Jilin-1 video satellite images exhibit superior reconstruction performance when compared with the state-of-the-art SR approaches.

[1]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Rui Bian,et al.  A Novel Stochastic Stratified Average Gradient Method: Convergence Rate and Its Complexity , 2017, 2018 International Joint Conference on Neural Networks (IJCNN).

[3]  Jürgen Schmidhuber,et al.  Training Very Deep Networks , 2015, NIPS.

[4]  Junjun Jiang,et al.  Edge-Enhanced GAN for Remote Sensing Image Superresolution , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[5]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Thomas R. Walter,et al.  Radar Path Delay Effects in Volcanic Gas Plumes: The Case of Láscar Volcano, Northern Chile , 2018, Remote. Sens..

[7]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[10]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Stefan Harmeling,et al.  Image denoising: Can plain neural networks compete with BM3D? , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Jiayi Ma,et al.  Hyperspectral Unmixing with Robust Collaborative Sparse Regression , 2016, Remote. Sens..

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Shuiping Gou,et al.  Remote Sensing Image Super-Resolution Reconstruction Based on Nonlocal Pairwise Dictionaries and Double Regularization , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Karen Anderson,et al.  Tracking Fine-Scale Structural Changes in Coastal Dune Morphology Using Kite Aerial Photography and Uncertainty-Assessed Structure-from-Motion Photogrammetry , 2018, Remote. Sens..

[20]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Tong Tong,et al.  Image Super-Resolution Using Dense Skip Connections , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.