Strengthen the Feature Distinguishability of Geo-Object Details in the Semantic Segmentation of High-Resolution Remote Sensing Images

Semantic segmentation is one of the hot topics in the field of remote sensing image intelligent analysis. Deep convolutional neural network (DCNN) has become a mainstream technology in semantic segmentation due to its powerful semantic feature representation. The emergence of high-resolution remote sensing imagery has provided massive detail information, but difficulties and challenges remain in the “feature representation of fine geo objects” and “feature distinction of easily confusing geo objects.” To this end, this article focuses on the distinguishing features of geo-object details and proposes a novel DCNN-based semantic segmentation. First, the cascaded relation attention module is adopted to determine the relationship among different channels or positions. Then, information connection and error correction are used to capture and fuse the features of geo-object details. The output feature representations are provided by the multiscale feature module. Besides, the proposed model uses the boundary affinity loss to gain accurate and clear geo-object boundary. The experimental results on the Potsdam and Vaihingen datasets demonstrate that the proposed model can achieve excellent segmentation performance on overall accuracy and mean intersection over union. Furthermore, the results of ablation and visualization analyses also verify the feasibility and effectiveness of the proposed method.

[1]  Jie Chen,et al.  SMAF-Net: Sharing Multiscale Adversarial Feature for High-Resolution Remote Sensing Imagery Semantic Segmentation , 2021, IEEE Geoscience and Remote Sensing Letters.

[2]  Xiao Xiang Zhu,et al.  Learning to Pay Attention on Spectral Domain: A Spectral Attention Module-Based Convolutional Network for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[3]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[4]  Cheolhee Yoo,et al.  Comparison between convolutional neural networks and random forest for local climate zone classification in mega urban areas using Landsat images , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[5]  Xiaojuan Qi,et al.  ICNet for Real-Time Semantic Segmentation on High-Resolution Images , 2017, ECCV.

[6]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[7]  Lingfeng Wang,et al.  Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Network , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[8]  Yuhao Wang,et al.  Dense Semantic Labeling with Atrous Spatial Pyramid Pooling and Decoder for High-Resolution Remote Sensing Imagery , 2018, Remote. Sens..

[9]  Yufeng Wang,et al.  ERN: Edge Loss Reinforced Semantic Segmentation Network for Remote Sensing Images , 2018, Remote. Sens..

[10]  Fan Zhang,et al.  TreeUNet: Adaptive Tree convolutional neural networks for subdecimeter aerial image segmentation , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[11]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Qian Shi,et al.  Local climate zone mapping as remote sensing scene classification using deep learning: A case study of metropolitan China , 2020 .

[13]  Kun Yu,et al.  DenseASPP for Semantic Segmentation in Street Scenes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Jingdong Wang,et al.  OCNet: Object Context Network for Scene Parsing , 2018, ArXiv.

[16]  Min Wang,et al.  Context Aggregation Network for Semantic Labeling in Aerial Images , 2019, Remote. Sens..

[17]  Stella X. Yu,et al.  Adaptive Affinity Fields for Semantic Segmentation , 2018, ECCV.

[18]  Xinying Xu,et al.  Automatic Image Segmentation With Superpixels and Image-Level Labels , 2019, IEEE Access.

[19]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Xuejian Li,et al.  Very High Resolution Remote Sensing Imagery Classification Using a Fusion of Random Forest and Deep Learning Technique—Subtropical Area for Example , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Meng Lu,et al.  Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Jian Yao,et al.  RoadNet: Learning to Comprehensively Analyze Road Networks in Complex Urban Scenes From High-Resolution Remotely Sensed Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yi Wang,et al.  Multiple Kernel-Based SVM Classification of Hyperspectral Images by Combining Spectral, Spatial, and Semantic Information , 2020, Remote. Sens..

[27]  Nicolas Brown,et al.  Mapping dead forest cover using a deep convolutional neural network and digital aerial photography , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[28]  Jie Chen,et al.  SPMF-Net: Weakly Supervised Building Segmentation by Combining Superpixel Pooling and Multi-Scale Feature Fusion , 2020, Remote. Sens..

[29]  Hangbin Wu,et al.  Urban Land Cover Classification of High-Resolution Aerial Imagery Using a Relation-Enhanced Multiscale Convolutional Network , 2020, Remote. Sens..

[30]  Emanuele Frontoni,et al.  Deep Learning for Soil and Crop Segmentation from Remotely Sensed Data , 2019, Remote. Sens..

[31]  Jiankun Hu,et al.  Multi-Resolution Weed Classification via Convolutional Neural Network and Superpixel Based Local Binary Pattern Using Remote Sensing Images , 2019, Remote. Sens..

[32]  Yang Zhao,et al.  Deep High-Resolution Representation Learning for Visual Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Lorenzo Bruzzone,et al.  Multi-Scale Context Aggregation for Semantic Segmentation of Remote Sensing Images , 2020, Remote. Sens..

[34]  Pengcheng Zhang,et al.  Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[35]  Li Pan,et al.  Dual Learning-Based Siamese Framework for Change Detection Using Bi-Temporal VHR Optical Remote Sensing Images , 2019, Remote. Sens..

[36]  Raymond Y. K. Lau,et al.  Road Detection and Centerline Extraction Via Deep Recurrent Convolutional Neural Network U-Net , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Jocelyn Chanussot,et al.  Dynamic Multicontext Segmentation of Remote Sensing Images Based on Convolutional Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Xuelong Li,et al.  Scene Classification With Recurrent Attention of VHR Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Yongjun Zhang,et al.  A Lightweight and Discriminative Model for Remote Sensing Scene Classification With Multidilation Pooling Module , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[41]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[42]  Jian Sun,et al.  DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Enrico Magli,et al.  Learning and Adapting Robust Features for Satellite Image Segmentation on Heterogeneous Data Sets , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Gang Yu,et al.  BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation , 2018, ECCV.

[45]  Hong Liu,et al.  Expectation-Maximization Attention Networks for Semantic Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.