Semantic Segmentation of High Resolution Remote Sensing Images with Extra Context Attention Mechanism

High Resolution Remote Sensing Images (HRRSIs) usually have a larger size compared with natural images. Because of the limitation of GPU memory, it is not possible to train semantic segmentation models on HRRSIs directly. Commonly used methodologies perform training and prediction on cropped sub-images. Thus they fail to model potential dependencies between pixels beyond sub-images. To solve this problem, we firstly propose extra context attention to capture global information from larger receptive fields and discriminative information from surrounding pixels beyond sub-images. Secondly, we apply feature map refinement module to better fuse extra context information and primary semantic information. Finally, we apply channel attention module to improve the performance of the decoder so that features from different levels can be better integrated. Experimental results on ISPRS Potsdam dataset demonstrate the effectiveness of our proposed network for semantic segmentation in HRRSIs.

[1]  Markus Gerke,et al.  The ISPRS benchmark on urban object classification and 3D building reconstruction , 2012 .

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[5]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[8]  Michele Volpi,et al.  Dense Semantic Labeling of Subdecimeter Resolution Images With Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Kun Yu,et al.  DenseASPP for Semantic Segmentation in Street Scenes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[12]  Lingfeng Wang,et al.  Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Network , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[13]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Lorenzo Bruzzone,et al.  A Deep Architecture Based on a Two-Stage Learning for Semantic Segmentation of Large-Size Remote Sensing Images , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[15]  Lele Cao.  Fine-Grained Road Mining from Satellite Images with Bilateral Xception and DeepLab , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[16]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Feng Wang,et al.  Semantic Segmentation of High Resolution Remote Sensing Image Based on Batch-Attention Mechanism , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[18]  Yunchao Wei,et al.  CCNet: Criss-Cross Attention for Semantic Segmentation. , 2020, IEEE transactions on pattern analysis and machine intelligence.

[19]  Zhenzhong Chen,et al.  Superpixel-enhanced deep neural forest for remote sensing image semantic segmentation , 2020 .

[20]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.