Automatic Building Extraction via Adaptive Iterative Segmentation With LiDAR Data and High Spatial Resolution Imagery Fusion

Extracting buildings from remotely sensed data is a fundamental task in many geospatial applications. However, this task is resistant to automation due to variability in building shapes and the environmental complexity surrounding buildings. To solve this problem, this article introduces a novel automatic building extraction method that integrates LiDAR data and high spatial resolution imagery using adaptive iterative segmentation and hierarchical overlay analysis based on data fusion. An adaptive iterative segmentation method overcomes over- and undersegmentation based on the globalized probability of boundary contour detection algorithm. A data-fusion-based hierarchical overlay analysis extracts building candidate regions based on segmentation results. A morphological operation optimizes a building candidate region to obtain final building results. Experiments were conducted on the international society for photogrammetry and remote sensing (ISPRS) Vaihingen benchmark dataset. The extracted building footprints were compared with those extracted using the state-of-the-art methods. Evaluation results show that the proposed method achieved the highest area-based quality compared to results from the other tested methods on the ISPRS website. A detailed comparison with four state-of-the-art methods shows that the proposed method requiring no samples achieves competitive extraction results. Furthermore, the proposed method achieved a completeness of 94.1%, a correctness of 90.3%, and a quality of 85.5% over the whole Vaihingen dataset, indicating that the method is robust, with great potential in practical applications.

[1]  C. Fraser,et al.  Automatic Detection of Residential Buildings Using LIDAR Data and Multispectral Imagery , 2010 .

[2]  Ramakant Nevatia,et al.  Detection and Modeling of Buildings from Multiple Aerial Images , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[4]  Hans-Gerd Maas,et al.  Geometric Refinement of ALS-Data Derived Building Models Using Monoscopic Aerial Images , 2017, Remote. Sens..

[5]  Pablo Andrés Arbeláez,et al.  Boundary Extraction in Natural Images Using Ultrametric Contour Maps , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[6]  Li Yong,et al.  ADAPTIVE BUILDING EDGE DETECTION BY COMBINING LIDAR DATA AND AERIAL IMAGES , 2008 .

[7]  Michael Werman,et al.  Fusing Time-of-Flight Depth and Color for Real-Time Segmentation and Tracking , 2009, Dyn3D.

[8]  G. Alonso Segmentación de Imágenes con Algoritmos de Agrupamiento Utilizando la Base de Datos BSDS500 "The Berkeley Segmentation Dataset and Benchmark , 2016 .

[9]  Stefanos Kollias,et al.  Unsupervised semantic object segmentation of stereoscopic video sequences , 1999, Proceedings 1999 International Conference on Information Intelligence and Systems (Cat. No.PR00446).

[10]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[11]  Ali Mohammadzadeh,et al.  A Novel Building and Tree Detection Method From LiDAR Data and Aerial Images , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[13]  Takayoshi Yamashita,et al.  Multiple Object Extraction from Aerial Imagery with Convolutional Neural Networks , 2016, IRIACV.

[14]  Rongjun Qin,et al.  A Hierarchical Building Detection Method for Very High Resolution Remotely Sensed Images Combined with DSM Using Graph Cut Optimization , 2014 .

[15]  Xiaoling Chen,et al.  BUILDING BOUNDARY EXTRACTION FROM HIGH RESOLUTION IMAGERY AND LIDAR DATA , 2008 .

[16]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  John Trinder,et al.  Using the Dempster-Shafer method for the fusion of LIDAR data and multi-spectral images for building detection , 2005, Inf. Fusion.

[18]  Pengcheng Zhang,et al.  Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[19]  I. Dowman,et al.  Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction * , 2007 .

[20]  Franz Rottensteiner,et al.  Automatic Generation of High-Quality Building Models from Lidar Data , 2003, IEEE Computer Graphics and Applications.

[21]  S. Christensen,et al.  Colour and shape analysis techniques for weed detection in cereal fields , 2000 .

[22]  Jiangye Yuan,et al.  Learning Building Extraction in Aerial Scenes with Convolutional Networks , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Masashi Matsuoka,et al.  Multi-scale solution for building extraction from LiDAR and image data , 2009, Int. J. Appl. Earth Obs. Geoinformation.

[24]  Markus Gerke,et al.  The ISPRS benchmark on urban object classification and 3D building reconstruction , 2012 .

[25]  Kyoung Mu Lee,et al.  Fusion of Lidar and Imagery for Reliable Building Extraction , 2008 .

[26]  Jing Xiao,et al.  Fusion of airborne laserscanning point clouds and images for supervised and unsupervised scene classification , 2014 .

[27]  Siyang Chen,et al.  Automatic building extraction from LiDAR data fusion of point and grid-based features , 2017 .

[28]  Wen-Hsiang Tsai,et al.  Moment-preserving thresolding: A new approach , 1985, Comput. Vis. Graph. Image Process..

[29]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jixian Zhang Multi-source remote sensing data fusion: status and trends , 2010 .

[31]  Mohamad M. Awad Toward Robust Segmentation Results Based on Fusion Methods for Very High Resolution Optical Image and LiDAR Data , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[32]  Ling Shao,et al.  Structure-Preserving Binary Representations for RGB-D Action Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  J. A. Schell,et al.  Monitoring vegetation systems in the great plains with ERTS , 1973 .

[34]  Shiyong Cui,et al.  Building Footprint Extraction From VHR Remote Sensing Images Combined With Normalized DSMs Using Fused Fully Convolutional Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Pascal Kosuth,et al.  Influence of training sampling protocol and of feature space optimization methods on supervised classification results , 2007, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[36]  Guojun Lu,et al.  An Automatic Building Extraction and Regularisation Technique Using LiDAR Point Cloud Data and Orthoimage , 2016, Remote. Sens..

[37]  Yan Meng,et al.  Subtracted Histogram: Utilizing Mutual Relation Between Features for Thresholding , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Nikolaos Doulamis,et al.  Building Extraction From LiDAR Data Applying Deep Convolutional Neural Networks , 2019, IEEE Geoscience and Remote Sensing Letters.

[39]  Jonathan T. Barron,et al.  Multiscale Combinatorial Grouping , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Yong Li,et al.  An improved building boundary extraction algorithm based on fusion of optical imagery and LIDAR data , 2013 .

[41]  C. Fraser,et al.  Building detection in complex scenes thorough effective separation of buildings from trees , 2012 .

[42]  Michael Ying Yang,et al.  Contour Detection for UAV-Based Cadastral Mapping , 2017, Remote. Sens..

[43]  Charless C. Fowlkes,et al.  Contour Detection and Hierarchical Image Segmentation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Nikolaos Doulamis,et al.  Deep convolutional neural networks for building extraction from orthoimages and dense image matching point clouds , 2017 .

[45]  C. Briese,et al.  A NEW METHOD FOR BUILDING EXTRACTION IN URBAN AREAS FROM HIGH-RESOLUTION LIDAR DATA , 2002 .

[46]  Liangpei Zhang,et al.  Morphological Building/Shadow Index for Building Extraction From High-Resolution Imagery Over Urban Areas , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Qian Zhang,et al.  Classification of Ultra-High Resolution Orthophotos Combined with DSM Using a Dual Morphological Top Hat Profile , 2015, Remote. Sens..

[48]  N. El-Sheimy,et al.  A NEW OBJECT BASED METHOD FOR AUTOMATED EXTRACTION OF URBAN OBJECTS FROM AIRBORNE SENSORS DATA , 2012 .

[49]  Lei Chen,et al.  Building detection in an urban area using lidar data and QuickBird imagery , 2012 .

[50]  Youqiang Dong,et al.  Extraction of Buildings from Multiple-View Aerial Images Using a Feature-Level-Fusion Strategy , 2018, Remote Sensing.

[51]  Ting Chen,et al.  Building Extraction from UAV Images Jointly Using 6D-SLIC and Multiscale Siamese Convolutional Networks , 2019, Remote. Sens..

[52]  Michael Cramer,et al.  The DGPF-Test on Digital Airborne Camera Evaluation - Over- view and Test Design , 2010 .

[53]  C. Song,et al.  Urban 3D GIS From LiDAR and digital aerial images , 2004, Comput. Geosci..

[54]  Yongyang Xu,et al.  Building Extraction in Very High Resolution Remote Sensing Imagery Using Deep Learning and Guided Filters , 2018, Remote. Sens..

[55]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[56]  Daniel Cremers,et al.  FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture , 2016, ACCV.

[57]  P. Zingaretti,et al.  Performance evaluation of automated approaches to building detection in multi-source aerial data , 2010 .

[58]  Xin Huang,et al.  A Multidirectional and Multiscale Morphological Index for Automatic Building Extraction from Multispectral GeoEye-1 Imagery , 2011 .

[59]  Markus Vincze,et al.  Multimodal cue integration through Hypotheses Verification for RGB-D object recognition and 6DOF pose estimation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[60]  D. Grigillo,et al.  Automatic extraction and building change detection from digital surface model and multispectral orthophoto , 2011 .

[61]  Jaewook Jung,et al.  Results of the ISPRS benchmark on urban object detection and 3D building reconstruction , 2014 .