Building Footprint Extraction From Unmanned Aerial Vehicle Images Via PRU-Net: Application to Change Detection

As the manual detection of building footprint is inefficient and labor-intensive, this study proposed a method of building footprint extraction and change detection based on deep convolutional neural networks. The study modified the existing U-Net model to develop the “PRU-Net” model. PRU-Net incorporates pyramid scene parsing (PSP) to allow multiscale scene parsing, a residual block (RB) in ResNet for feature extraction, and focal loss to address sample imbalance. Within the proposed method, building footprint extraction is conducted as follows: 1) unmanned aerial vehicle images are cropped, denoised, and semantically marked, and datasets are created (including training/validation and prediction datasets); 2) the training/validation and prediction datasets are input into the full convolutional neural network PRU-Net for model training/validation and prediction. Compared with the U-Net, PSP+U-Net (PU-Net), and U-Net++ models, PRU-Net offers improved footprint extraction of buildings with a range of sizes and shapes. The large-scale experimental results demonstrated the effectiveness of the PSP module for multiscale scene analysis and the RB module for feature extraction. After demonstrating the improvements in building extraction offered by PRU-Net, the building footprint results were further processed to generate a building change map.

[1]  Wei Liu,et al.  Mapping 10-m Resolution Rural Settlements Using Multi-Source Remote Sensing Datasets with the Google Earth Engine Platform , 2020, Remote. Sens..

[2]  Yao Yu,et al.  Ensemble Learning for Hyperspectral Image Classification Using Tangent Collaborative Representation , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Xiao Xiang Zhu,et al.  Building Footprint Generation by Integrating Convolution Neural Network With Feature Pairwise Conditional Random Field (FPCRF) , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Mohammad Sohel Rahman,et al.  MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation , 2019, Neural Networks.

[5]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Wei Liu,et al.  Accurate Building Extraction from Fused DSM and UAV Images Using a Chain Fully Convolutional Neural Network , 2019, Remote. Sens..

[7]  Payman Moallem,et al.  Transferable Object-Based Framework Based on Deep Convolutional Neural Networks for Building Extraction , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Masashi Matsuoka,et al.  Newly Built Construction Detection in SAR Images Using Deep Learning , 2019, Remote. Sens..

[9]  Pengcheng Zhang,et al.  Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[10]  Francesca Bovolo,et al.  An Approach to Multiple Change Detection in VHR Optical Images Based on Iterative Clustering and Adaptive Thresholding , 2019, IEEE Geoscience and Remote Sensing Letters.

[11]  Francesca Bovolo,et al.  Unsupervised Deep Change Vector Analysis for Multiple-Change Detection in VHR Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Yun Zhang,et al.  Multispectral change detection using multivariate Kullback-Leibler distance , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[13]  Xiao Xiang Zhu,et al.  Building Footprint Generation Using Improved Generative Adversarial Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[14]  Meng Lu,et al.  Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Ruofei Zhong,et al.  Multilevel Building Detection Framework in Remote Sensing Images Based on Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[17]  Jiangye Yuan,et al.  Building Extraction at Scale Using Convolutional Neural Network: Mapping of the United States , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Jianxin Wu,et al.  Vortex Pooling: Improving Context Representation in Semantic Segmentation , 2018, ArXiv.

[19]  Yongjun Zhang,et al.  Object-Based Change Detection for VHR Images Based on Multiscale Uncertainty Analysis , 2018, IEEE Geoscience and Remote Sensing Letters.

[20]  Wei Lee Woon,et al.  Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks , 2017 .

[21]  A. Belotserkovsky,et al.  Building detection on aerial and space images , 2017, 2017 International Conference on Information and Digital Technologies (IDT).

[22]  Qian Du,et al.  Multiscale Morphological Compressed Change Vector Analysis for Unsupervised Multiple Change Detection , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[23]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[24]  Jian Liang,et al.  A Bottom-Up/Top-Down Hybrid Algorithm for Model-Based Building Detection in Single Very High Resolution SAR Image , 2017, IEEE Geoscience and Remote Sensing Letters.

[25]  Hannes Taubenböck,et al.  Unsupervised change detection in VHR remote sensing imagery - an object-based clustering approach in a dynamic urban environment , 2017, Int. J. Appl. Earth Obs. Geoinformation.

[26]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[28]  Maggi Kelly,et al.  Change detection of built-up land: A framework of combining pixel-based detection and object-based recognition , 2016 .

[29]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[30]  Jiangye Yuan,et al.  Automatic Building Extraction in Aerial Scenes Using Convolutional Networks , 2016, ArXiv.

[31]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[33]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[34]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[35]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Liangpei Zhang,et al.  Building Change Detection From Multitemporal High-Resolution Remotely Sensed Images Based on a Morphological Building Index , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[38]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[39]  Mahamadou Idrissa,et al.  Building Change Detection from Uniform Regions , 2012, CIARP.

[40]  Zhanyi Hu,et al.  Urban change detection under large view and illumination variations , 2009 .

[41]  Mustafa Turker,et al.  Building‐based damage detection due to earthquake using the watershed segmentation of the post‐event aerial images , 2008 .