E-D-Net: Automatic Building Extraction From High-Resolution Aerial Images With Boundary Information

The automatic extraction of buildings from high-resolution aerial imagery plays a significant role in many urban applications. Recently, the convolution neural network (CNN) has gained much attention in remote sensing field and achieved a remarkable performance in building segmentation from visible aerial images. However, most of the existing CNN-based methods still have the problem of tending to produce predictions with poor boundaries. To address this problem, in this article, a novel semantic segmentation neural network named edge-detail-network (E-D-Net) is proposed for building segmentation from visible aerial images. The proposed E-D-Net consists of two subnetworks E-Net and D-Net. On the one hand, E-Net is designed to capture and preserve the edge information of the images. On the other hand, D-Net is designed to refine the results of E-Net and get a prediction with higher detail quality. Furthermore, a novel fusion strategy, which combines the outputs of the two subnetworks is proposed to integrate edge information with fine details. Experimental results on the INRIA aerial image labeling dataset and the ISPRS Vaihingen 2-D semantic labeling dataset demonstrate that, compared with the existing CNN-based model, the proposed E-D-Net provides noticeably more robust and higher building extraction performance, thus making it a useful tool for practical application scenarios.

[1]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[2]  Pierre Alliez,et al.  Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[3]  Kyoung Mu Lee,et al.  Fusion of Lidar and Imagery for Reliable Building Extraction , 2008 .

[4]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[5]  Igor Sevo,et al.  Convolutional Neural Network Based Automatic Object Detection on Aerial Images , 2016, IEEE Geoscience and Remote Sensing Letters.

[6]  Guoqiang Han,et al.  R³Net: Recurrent Residual Refinement Network for Saliency Detection , 2018, IJCAI.

[7]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Xiaoxiao Li,et al.  Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Xiang Li,et al.  Building-A-Nets: Robust Building Extraction From High-Resolution Remote Sensing Images With Adversarial Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  Wenzhong Shi,et al.  Automatic Building Extraction via Adaptive Iterative Segmentation With LiDAR Data and High Spatial Resolution Imagery Fusion , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[13]  Markus Gerke,et al.  Use of the stair vision library within the ISPRS 2D semantic labeling benchmark (Vaihingen) , 2014 .

[14]  Jiangye Yuan,et al.  Building Extraction at Scale Using Convolutional Neural Network: Mapping of the United States , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Takayoshi Yamashita,et al.  Multiple Object Extraction from Aerial Imagery with Convolutional Neural Networks , 2016, IRIACV.

[16]  Huiyu Zhou,et al.  Airport Detection in SAR Images Via Salient Line Segment Detector and Edge-Oriented Region Growing , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Alexey Shvets,et al.  TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentation , 2018, Computer-Aided Analysis of Gastrointestinal Videos.

[18]  Andreas Dengel,et al.  Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks , 2017, 2019 IEEE International Conference on Image Processing (ICIP).

[19]  Yinghua Ye,et al.  A Deep Learning Approach on Building Detection from Unmanned Aerial Vehicle-Based Images in Riverbank Monitoring , 2018, Sensors.

[20]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[21]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  J. R. Jensen,et al.  Remote Sensing of Urban/Suburban Infrastructure and Socio‐Economic Attributes , 2011 .

[23]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[24]  Nikhil R. Pal,et al.  On minimum cross-entropy thresholding , 1996, Pattern Recognit..

[25]  Han Jiang,et al.  Fully convolutional networks for building and road extraction: Preliminary results , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[26]  Xuejin Chen,et al.  HF-FCN: Hierarchically Fused Fully Convolutional Network for Robust Building Extraction , 2016, ACCV.

[27]  Geoffrey E. Hinton,et al.  Learning to Detect Roads in High-Resolution Aerial Images , 2010, ECCV.

[28]  Eugenio Culurciello,et al.  LinkNet: Exploiting encoder representations for efficient semantic segmentation , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[29]  Xinchang Zhang,et al.  Arbitrary-Shaped Building Boundary-Aware Detection with Pixel Aggregation Network , 2020 .

[30]  M. Ruiz Espejo Sampling , 2013, Encyclopedic Dictionary of Archaeology.

[31]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[32]  Yongchao Gong,et al.  Mask Scoring R-CNN , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Jun Zhu,et al.  Refined Extraction Of Building Outlines From High-Resolution Remote Sensing Imagery Based on a Multifeature Convolutional Neural Network and Morphological Filtering , 2020, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[34]  Maoguo Gong,et al.  A Deep Convolutional Coupling Network for Change Detection Based on Heterogeneous Optical and Radar Images , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Jordan M. Malof,et al.  Large-Scale Semantic Classification: Outcome of the First Year of Inria Aerial Image Labeling Benchmark , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[36]  Xiaopeng Zhang,et al.  Building Extraction from Remotely Sensed Images by Integrating Saliency Cue , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[37]  Fang Liu,et al.  SAR image despeckling via bilateral filtering , 2009 .