Semantic Segmentation-Based Building Footprint Extraction Using Very High-Resolution Satellite Images and Multi-Source GIS Data

Automatic extraction of building footprints from high-resolution satellite imagery has become an important and challenging research issue receiving greater attention. Many recent studies have explored different deep learning-based semantic segmentation methods for improving the accuracy of building extraction. Although they record substantial land cover and land use information (e.g., buildings, roads, water, etc.), public geographic information system (GIS) map datasets have rarely been utilized to improve building extraction results in existing studies. In this research, we propose a U-Net-based semantic segmentation method for the extraction of building footprints from high-resolution multispectral satellite images using the SpaceNet building dataset provided in the DeepGlobe Satellite Challenge of IEEE Conference on Computer Vision and Pattern Recognition 2018 (CVPR 2018). We explore the potential of multiple public GIS map datasets (OpenStreetMap, Google Maps, and MapWorld) through integration with the WorldView-3 satellite datasets in four cities (Las Vegas, Paris, Shanghai, and Khartoum). Several strategies are designed and combined with the U-Net–based semantic segmentation model, including data augmentation, post-processing, and integration of the GIS map data and satellite images. The proposed method achieves a total F1-score of 0.704, which is an improvement of 1.1% to 12.5% compared with the top three solutions in the SpaceNet Building Detection Competition and 3.0% to 9.2% compared with the standard U-Net–based method. Moreover, the effect of each proposed strategy and the possible reasons for the building footprint extraction results are analyzed substantially considering the actual situation of the four cities.

[1]  Nikhil Ketkar,et al.  Deep Learning with Python , 2017 .

[2]  Wei Tu,et al.  Integrating Aerial and Street View Images for Urban Land Use Classification , 2018, Remote. Sens..

[3]  Jiangye Yuan,et al.  Learning to count buildings in diverse aerial scenes , 2014, SIGSPATIAL/GIS.

[4]  Mariana Belgiu,et al.  Comparing supervised and unsupervised multiresolution segmentation approaches for extracting buildings from very high resolution imagery , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[5]  Meng Lu,et al.  Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Geoffrey E. Hinton,et al.  Machine Learning for Aerial Image Labeling , 2013 .

[7]  Biao Wang,et al.  Building Extraction in Very High Resolution Imagery by Dense-Attention Networks , 2018, Remote. Sens..

[8]  Biswajeet Pradhan,et al.  A rule-based parameter aided with object-based classification approach for extraction of building and roads from WorldView-2 images , 2014 .

[9]  Yongyang Xu,et al.  Road Extraction from High-Resolution Remote Sensing Imagery Using Deep Learning , 2018, Remote. Sens..

[10]  Tingting Lv,et al.  Detecting Building Edges from High Spatial Resolution Remote Sensing Imagery Using Richer Convolution Features Network , 2018, Remote. Sens..

[11]  Meng Lu,et al.  A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery , 2018, International Journal of Remote Sensing.

[12]  Shiyong Cui,et al.  Building Change Detection Based on Satellite Stereo Imagery and Digital Surface Models , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Çaglar Senaras,et al.  Automated Detection of Arbitrarily Shaped Buildings in Complex Environments From Monocular VHR Optical Satellite Imagery , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[14]  Congcong Li,et al.  Stacked Autoencoder-based deep learning for remote-sensing image classification: a case study of African land-cover mapping , 2016 .

[15]  Jonathan Li,et al.  Object-Based Features for House Detection from RGB High-Resolution Images , 2018, Remote. Sens..

[16]  Xiang Li,et al.  Building-A-Nets: Robust Building Extraction From High-Resolution Remote Sensing Images With Adversarial Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[17]  Xinchang Zhang,et al.  Extracting Building Boundaries from High Resolution Optical Images and LiDAR Data by Integrating the Convolutional Neural Network and the Active Contour Model , 2018, Remote. Sens..

[18]  Yueyan Liu,et al.  Fully Connected Conditional Random Fields for High-Resolution Remote Sensing Land Use/Land Cover Classification with Convolutional Neural Networks , 2018, Remote. Sens..

[19]  Peijun Du,et al.  Object-Based Change Detection in Urban Areas from High Spatial Resolution Images Based on Multiple Features and Ensemble Learning , 2018, Remote. Sens..

[20]  Yanfei Liu,et al.  Scene Classification Based on a Deep Random-Scale Stretched Convolutional Neural Network , 2018, Remote. Sens..

[21]  Haohuan Fu,et al.  Semantic Segmentation Based Building Extraction Method Using Multi-source GIS Map Datasets and Satellite Imagery , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Leonardo Vanneschi,et al.  Improved Fully Convolutional Network with Conditional Random Fields for Building Extraction , 2018, Remote. Sens..

[23]  Yifan Wu,et al.  Aerial Imagery for Roof Segmentation: A Large-Scale Dataset towards Automatic Mapping of Buildings , 2018, ArXiv.

[24]  Shunichi Koshimura,et al.  Towards Operational Satellite-Based Damage-Mapping Using U-Net Convolutional Network: A Case Study of 2011 Tohoku Earthquake-Tsunami , 2018, Remote. Sens..

[25]  Markus Gerke,et al.  The ISPRS benchmark on urban object classification and 3D building reconstruction , 2012 .

[26]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[27]  Wei Lee Woon,et al.  Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks , 2017 .

[28]  Ali Ozgun Ok,et al.  Automated detection of buildings from single VHR multispectral images using shadow information and graph cuts , 2013 .

[29]  Chunhong Pan,et al.  Building extraction from multi-source remote sensing images via deep deconvolution neural networks , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[30]  Shihong Du,et al.  Semantic classification of urban buildings combining VHR image and GIS data: An improved random forest approach , 2015 .

[31]  Vladimir Iglovikov,et al.  Satellite Imagery Feature Detection using Deep Convolutional Neural Network: A Kaggle Competition , 2017, ArXiv.

[32]  Yongyang Xu,et al.  Building Extraction in Very High Resolution Remote Sensing Imagery Using Deep Learning and Guided Filters , 2018, Remote. Sens..

[33]  Bo Huang,et al.  Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery , 2018, Remote Sensing of Environment.

[34]  Adam Van Etten,et al.  SpaceNet: A Remote Sensing Dataset and Challenge Series , 2018, ArXiv.

[35]  Bertrand Le Saux,et al.  Joint Learning from Earth Observation and OpenStreetMap Data to Get Faster Better Semantic Maps , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[36]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Siyang Chen,et al.  Automatic building extraction from LiDAR data fusion of point and grid-based features , 2017 .

[38]  Lin Lei,et al.  Arbitrary-Oriented Vehicle Detection in Aerial Imagery with Single Convolutional Neural Networks , 2017, Remote. Sens..

[39]  Pierre Alliez,et al.  Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[40]  Kyoung Mu Lee,et al.  Fusion of Lidar and Imagery for Reliable Building Extraction , 2008 .

[41]  Guojun Lu,et al.  An Automatic Building Extraction and Regularisation Technique Using LiDAR Point Cloud Data and Orthoimage , 2016, Remote. Sens..

[42]  Cyrill Stachniss,et al.  WeedMap: A large-scale semantic weed mapping framework using aerial multispectral imaging and deep neural network for precision farming , 2018, Remote. Sens..

[43]  Jordan M. Malof,et al.  Large-Scale Semantic Classification: Outcome of the First Year of Inria Aerial Image Labeling Benchmark , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[44]  Eli Saber,et al.  Supervised Classification of Multisensor Remotely Sensed Images Using a Deep Learning Framework , 2018, Remote. Sens..

[45]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[46]  Zhenwei Shi,et al.  Maritime Semantic Labeling of Optical Remote Sensing Images with Multi-Scale Fully Convolutional Network , 2017, Remote. Sens..

[47]  Jing Huang,et al.  DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[48]  Lianru Gao,et al.  Semantic Labeling of High Resolution Aerial Imagery and LiDAR Data with Fine Segmentation Network , 2018, Remote. Sens..

[49]  Jiaojiao Tian,et al.  Spatiotemporal inferences for use in building detection using series of very-high-resolution space-borne stereo images , 2016 .

[50]  Haihong Zhu,et al.  A Multiple-Feature Reuse Network to Extract Buildings from Remote Sensing Imagery , 2018, Remote. Sens..

[51]  Junwei Han,et al.  A Survey on Object Detection in Optical Remote Sensing Images , 2016, ArXiv.

[52]  Weijia Li,et al.  Large-Scale Oil Palm Tree Detection from High-Resolution Satellite Images Using Two-Stage Convolutional Neural Networks , 2018, Remote. Sens..

[53]  Ye Zhang,et al.  A light and faster regional convolutional neural network for object detection in optical remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[54]  C. Fraser,et al.  Automatic Detection of Residential Buildings Using LIDAR Data and Multispectral Imagery , 2010 .

[55]  O. Tournaire,et al.  An efficient stochastic approach for building footprint extraction from digital elevation models , 2010 .

[56]  Ryosuke Shibasaki,et al.  A Boundary Regulated Network for Accurate Roof Segmentation and Outline Extraction , 2018, Remote. Sens..

[57]  K. Moffett,et al.  Remote Sens , 2015 .

[58]  Weijia Li,et al.  Deep Learning Based Oil Palm Tree Detection and Counting for High-Resolution Remote Sensing Images , 2016, Remote. Sens..

[59]  Yanming Chen,et al.  Building Point Detection from Vehicle-Borne LiDAR Data Based on Voxel Group and Horizontal Hollow Analysis , 2016, Remote. Sens..

[60]  Liangpei Zhang,et al.  Morphological Building/Shadow Index for Building Extraction From High-Resolution Imagery Over Urban Areas , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[61]  Bertrand Le Saux,et al.  Segment-before-Detect: Vehicle Detection and Classification through Semantic Segmentation of Aerial Images , 2017, Remote. Sens..

[62]  I. Dowman,et al.  Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction * , 2007 .