Refined Extraction Of Building Outlines From High-Resolution Remote Sensing Imagery Based on a Multifeature Convolutional Neural Network and Morphological Filtering

The automatic extraction of building outlines from high-resolution images is an important and challenging task. Convolutional neural networks have shown excellent results compared with traditional building extraction methods because of their ability to extract high-level abstract features from images. However, it is difficult to fully utilize the multiple features of current building extraction methods; consequently, the resulting building boundaries are irregular. To overcome these limitations, we propose a method for extracting buildings from high-resolution images using a multifeature convolutional neural network (MFCNN) and morphological filtering. Our method consists of two steps. First, the MFCNN, which consists of residual connected unit, dilated perception unit, and pyramid aggregation unit, is used to achieve pixel-level segmentation of the buildings. Second, morphological filtering is used to optimize the building boundaries, improve the boundary regularity, and obtain refined building boundaries. The Massachusetts and Inria datasets are selected for experimental analysis. Under the same experimental conditions, the extraction results achieved with the proposed MFCNN are compared with the results of other deep learning models that have been commonly used in recent years: FCN-8s, SegNet, and U-Net. The results on both datasets reveal that the proposed model improves the F1-score by 3.31%–5.99%, increases the overall accuracy (OA) by 1.85%–3.07%, and increases the intersection over union (IOU) by 3.47%–8.82%. These findings illustrate that the proposed method is effective at extracting buildings from complex scenes.

[1]  Clive S. Fraser,et al.  An Automatic and Threshold-Free Performance Evaluation System for Building Extraction Techniques From Airborne LIDAR Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  William J. Emery,et al.  Very High Resolution Multiangle Urban Classification Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Joonwhoan Lee,et al.  An Efficient Residual Learning Neural Network for Hyperspectral Image Superresolution , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[6]  J. Peng,et al.  Model and context‐driven building extraction in dense urban aerial images , 2005 .

[7]  Dumitru Erhan,et al.  Scalable Object Detection Using Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Hui Lin,et al.  Mapping urban impervious surface with dual-polarimetric SAR data: An improved method , 2016 .

[9]  Liangpei Zhang,et al.  A New Building Extraction Postprocessing Framework for High-Spatial-Resolution Remote-Sensing Imagery , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Lutz Gross,et al.  ARC-Net: An Efficient Network for Building Extraction From High-Resolution Aerial Images , 2019, IEEE Access.

[12]  N. Kanopoulos,et al.  Design of an image edge detection filter using the Sobel operator , 1988 .

[13]  Dong Yu,et al.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[14]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[15]  Wei Lee Woon,et al.  Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks , 2017 .

[16]  Lutz Gross,et al.  Automatic Building Extraction on High-Resolution Remote Sensing Imagery Using Deep Convolutional Encoder-Decoder With Spatial Pyramid Pooling , 2019, IEEE Access.

[17]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Dengsheng Lu,et al.  Regional mapping of human settlements in southeastern China with multisensor remotely sensed data , 2008 .

[19]  Jordi Inglada,et al.  Automatic recognition of man-made objects in high resolution optical remote sensing images by SVM classification of geometric image features , 2007 .

[20]  Qiming Qin,et al.  Effective Building Extraction From High-Resolution Remote Sensing Images With Multitask Driven Deep Neural Network , 2019, IEEE Geoscience and Remote Sensing Letters.

[21]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[22]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[23]  Payman Moallem,et al.  Transferable Object-Based Framework Based on Deep Convolutional Neural Networks for Building Extraction , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[24]  Weipeng Jing,et al.  ESFNet: Efficient Network for Building Extraction From High-Resolution Aerial Images , 2019, IEEE Access.

[25]  Pierre Alliez,et al.  Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[26]  Ali Ozturk,et al.  Automatic building extraction from very high-resolution image and LiDAR data with SVM algorithm , 2016, Arabian Journal of Geosciences.

[27]  Wei Su,et al.  Textural and local spatial statistics for the object‐oriented classification of urban areas using high resolution imagery , 2008 .

[28]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Xin Huang,et al.  A Multidirectional and Multiscale Morphological Index for Automatic Building Extraction from Multispectral GeoEye-1 Imagery , 2011 .

[32]  Helmut Mayer,et al.  Automatic Object Extraction from Aerial Imagery - A Survey Focusing on Buildings , 1999, Comput. Vis. Image Underst..

[33]  Hui Lin,et al.  Using pan-sharpened high resolution satellite data to improve impervious surfaces estimation , 2017, Int. J. Appl. Earth Obs. Geoinformation.

[34]  Motaz El-Saban,et al.  Automatic Pixelwise Object Labeling for Aerial Imagery Using Stacked U-Nets , 2018, ArXiv.

[35]  Jiangye Yuan,et al.  Learning Building Extraction in Aerial Scenes with Convolutional Networks , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[37]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[38]  Parvaneh Saeedi,et al.  Automatic Rooftop Extraction in Nadir Aerial Imagery of Suburban Regions Using Corners and Variational Level Set Evolution , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Lin Lei,et al.  Multi-scale object detection in remote sensing imagery with convolutional neural networks , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[40]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[41]  Andreas Dengel,et al.  Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks , 2017, 2019 IEEE International Conference on Image Processing (ICIP).

[42]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[43]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[44]  Shiyong Cui,et al.  Building Footprint Extraction From VHR Remote Sensing Images Combined With Normalized DSMs Using Fused Fully Convolutional Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[45]  Jiangye Yuan,et al.  Automatic Building Extraction in Aerial Scenes Using Convolutional Networks , 2016, ArXiv.

[46]  Geoffrey E. Hinton,et al.  Machine Learning for Aerial Image Labeling , 2013 .

[47]  Meng Lu,et al.  A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery , 2018, International Journal of Remote Sensing.

[48]  Hossein Arefi,et al.  A CNN-Based Approach for Automatic Building Detection and Recognition of Roof Types Using a Single Aerial Image , 2018, PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science.

[49]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Ruofei Zhong,et al.  Multilevel Building Detection Framework in Remote Sensing Images Based on Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[51]  Ramakant Nevatia,et al.  Detecting buildings in aerial images , 1988, Comput. Vis. Graph. Image Process..

[52]  Xiang Li,et al.  Building-A-Nets: Robust Building Extraction From High-Resolution Remote Sensing Images With Adversarial Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[53]  Jaewook Jung,et al.  Building Extraction from Satellite Images Using Mask R-CNN with Building Boundary Regularization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[54]  Yihua Tan,et al.  Automatic Extraction of Built-Up Areas From Panchromatic and Multispectral Remote Sensing Images Using Double-Stream Deep Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[55]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[56]  Qingshan Liu,et al.  Learning Multiscale Deep Features for High-Resolution Satellite Image Scene Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[57]  Yiming Yan,et al.  Building Extraction Based on an Optimized Stacked Sparse Autoencoder of Structure and Training Samples Using LIDAR DSM and Optical Images , 2017, Sensors.

[58]  Hai Wei,et al.  Fusion of LIDAR and aerial imagery for accurate building footprint extraction , 2009, Electronic Imaging.

[59]  Taejung Kim,et al.  Development of a graph-based approach for building detection , 1999, Image Vis. Comput..

[60]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[61]  J. Niemeyer,et al.  Contextual classification of lidar data and building object detection in urban areas , 2014 .

[62]  Xin Huang,et al.  Deep neural network for remote-sensing image interpretation: status and perspectives , 2019, National science review.