Gradient Prior Dilated Convolution Network for Remote Sensing Image Super-Resolution

Super-resolution (SR) aims to recover a high-resolution image from a single or multiple low-resolution images, compensating for the limitations of satellite sensor imaging. Deep convolutional neural networks have made great achievement in remote sensing image SR. In this article, we propose a novel gradient prior dilated convolutional network (GPDCN) for remote sensing image SR, which obtains contextual spatial connections and alleviates structural distortions. The GPDCN comprises a multiscale feature extraction network and a feature reconstruction network. The former employs a double-path dilated residual block with dilation convolution to increase a receptive field, a global self-attention module to detect long-range reliance among image patches, and a gradient propagation network to extract high-level gradient information. The latter uses the mixed high-order attention module to reconstruct the feature by collecting the high-order characteristics of multiple frequency bands. Experiments with the Massachusetts_Roads and 3K VEHICLE_SR datasets demonstrate that the GPDCN outperforms recent techniques concerning both quantitative and qualitative measures.

[1]  Haohuan Fu,et al.  RRSGAN: Reference-Based Super-Resolution for Remote Sensing Image , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Renlong Hang,et al.  Spectral Super-Resolution Network Guided by Intrinsic Properties of Hyperspectral Imagery , 2021, IEEE Transactions on Image Processing.

[3]  Kapil Mirchandani,et al.  DPSRGAN: Dilation Patch Super-Resolution Generative Adversarial Networks , 2021, 2021 6th International Conference for Convergence in Technology (I2CT).

[4]  Heng Tao Shen,et al.  Remote Sensing Image Super-Resolution via Mixed High-Order Attention Network , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Zhenwei Shi,et al.  Transformer-based Multi-Stage Enhancement for Remote Sensing Image Super-Resolution , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Wangmeng Zuo,et al.  Cross-Scale Internal Graph Neural Network for Image Super-Resolution , 2020, NeurIPS.

[7]  Jie Shao,et al.  Movie fill in the blank by joint learning from video and text with adaptive temporal attention , 2020, Pattern Recognit. Lett..

[8]  Jie Zhou,et al.  Structure-Preserving Super Resolution With Gradient Guidance , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Mingkui Tan,et al.  Closed-Loop Matters: Dual Regression Networks for Single Image Super-Resolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Bin Luo,et al.  Bidirectional Convolutional LSTM Neural Network for Remote Sensing Image Super-Resolution , 2019, Remote. Sens..

[12]  Dapeng Tao,et al.  Embedded Block Residual Network: A Recursive Restoration Model for Single-Image Super-Resolution , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Zongxu Pan,et al.  Super-Resolution of Single Remote Sensing Image Based on Residual Dense Backprojection Networks , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Weihong Deng,et al.  Mixed High-Order Attention Network for Person Re-Identification , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[15]  Xian Sun,et al.  Deep Residual Squeeze and Excitation Network for Remote Sensing Image Super-Resolution , 2019, Remote. Sens..

[16]  Shu-Tao Xia,et al.  Second-Order Attention Network for Single Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yuan Yan Tang,et al.  Maximum Likelihood Estimation-Based Joint Sparse Representation for the Classification of Hyperspectral Remote Sensing Images , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Qingshan Liu,et al.  Pyramid Fully Convolutional Network for Hyperspectral and Multispectral Image Fusion , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[19]  Junjun Jiang,et al.  Edge-Enhanced GAN for Remote Sensing Image Superresolution , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Wen Gao,et al.  Depth Restoration From RGB-D Data via Joint Adaptive Regularization and Thresholding on Manifolds , 2019, IEEE Transactions on Image Processing.

[21]  Qian Du,et al.  Self-Paced Joint Sparse Representation for the Classification of Hyperspectral Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Ye Zhang,et al.  Vehicle Detection in High-Resolution Images Using Superpixel Segmentation and CNN Iteration Strategy , 2019, IEEE Geoscience and Remote Sensing Letters.

[23]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Wei Wang,et al.  Deep Learning for Single Image Super-Resolution: A Brief Review , 2018, IEEE Transactions on Multimedia.

[25]  Errui Ding,et al.  Compact Generalized Non-local Network , 2018, NeurIPS.

[26]  Shuicheng Yan,et al.  A2-Nets: Double Attention Networks , 2018, NeurIPS.

[27]  Kangfu Mei,et al.  Multi-scale Residual Network for Image Super-Resolution , 2018, ECCV.

[28]  Yu Qiao,et al.  ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks , 2018, ECCV Workshops.

[29]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[30]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[31]  Gang Li,et al.  Change Detection in Heterogenous Remote Sensing Images via Homogeneous Pixel Transformation , 2018, IEEE Transactions on Image Processing.

[32]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Qingxiang Wu,et al.  Image super-resolution using a dilated convolutional neural network , 2018, Neurocomputing.

[34]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Eirikur Agustsson,et al.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[37]  Zhenwei Shi,et al.  Super-Resolution for Remote Sensing Images via Local–Global Combined Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[38]  Shuicheng Yan,et al.  Deep Edge Guided Recurrent Residual Learning for Image Super-Resolution , 2016, IEEE Transactions on Image Processing.

[39]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[41]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Gellért Máttyus,et al.  Fast Multiclass Vehicle Detection on Aerial Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[45]  Narendra Ahuja,et al.  Single image super-resolution from transformed self-exemplars , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Horst Bischof,et al.  Fast and accurate image upscaling with super-resolution forests , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Truong Q. Nguyen,et al.  Single Image Superresolution Based on Gradient Profile Sharpness , 2015, IEEE Transactions on Image Processing.

[48]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[49]  Katsuya Kondo,et al.  Edge preserving super-resolution with details based on similar texture synthesis , 2014, 2014 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).

[50]  Luc Van Gool,et al.  Anchored Neighborhood Regression for Fast Example-Based Super-Resolution , 2013, 2013 IEEE International Conference on Computer Vision.

[51]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[52]  Harry Shum,et al.  Gradient Profile Prior and Its Applications in Image Super-Resolution and Enhancement , 2011, IEEE Transactions on Image Processing.

[53]  Stephen Lin,et al.  Super resolution using edge prior and single image detail synthesis , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[54]  Mei Han,et al.  SoftCuts: A Soft Edge Smoothness Prior for Color Image Super-Resolution , 2009, IEEE Transactions on Image Processing.

[55]  H. Shum,et al.  Image super-resolution using gradient profile prior , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Peter M. Atkinson,et al.  A linearised pixel-swapping method for mapping rural linear land cover features from fine spatial resolution remotely sensed imagery , 2007, Comput. Geosci..

[57]  Raanan Fattal,et al.  Image upsampling via imposed edge statistics , 2007, ACM Trans. Graph..

[58]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[59]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .