BuildSeg: A General Framework for the Segmentation of Buildings

—Building segmentation from aerial images and 3D laser scanning (LiDAR) is a challenging task due to the diversity of backgrounds, building textures, and image quality. While current research using different types of convolutional and trans-former networks has considerably improved the performance on this task, even more accurate segmentation methods for buildings are desirable for applications such as automatic mapping. In this study, we propose a general framework termed BuildSeg employing a generic approach that can be quickly applied to segment buildings. Different data sources were combined to increase generalization performance. The approach yields good results for different data sources as shown by experiments on high-resolution multi-spectral and LiDAR imagery of cities in Norway, Denmark and France. We applied ConvNeXt and SegFormer based models on the high resolution aerial image dataset from the MapAI-competition. The methods achieved an IOU of 0.7902 and a boundary IOU of 0.6185. We used post-processing to account for the rectangular shape of the objects. This increased the boundary IOU from 0.6185 to 0.6189.

[1]  Matthew S. Hagen,et al.  A Comprehensive Review of Modern Object Segmentation Approaches , 2023, Found. Trends Comput. Graph. Vis..

[2]  A. Nossum,et al.  MapAI: Precision in Building Segmentation , 2022, Nordic Machine Intelligence.

[3]  Trevor Darrell,et al.  A ConvNet for the 2020s , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Ribana Roscher,et al.  ArtifiVe-Potsdam: A Benchmark for Learning with Artificial Objects for Improved Aerial Vehicle Detection , 2021, 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS.

[5]  Anima Anandkumar,et al.  SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers , 2021, NeurIPS.

[6]  Ross B. Girshick,et al.  Boundary IoU: Improving Object-Centric Image Segmentation Evaluation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Zhouchen Lin,et al.  PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Maryam Rahnemoonfar,et al.  FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding , 2020, IEEE Access.

[9]  Ribana Roscher,et al.  SEMCITY TOULOUSE: A BENCHMARK FOR BUILDING INSTANCE SEGMENTATION IN SATELLITE IMAGES , 2020 .

[10]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[11]  Meng Lu,et al.  Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Pierre Alliez,et al.  Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Geoffrey E. Hinton,et al.  Machine Learning for Aerial Image Labeling , 2013 .

[16]  Giampaolo Ferraioli,et al.  Multichannel InSAR Building Edge Detection , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Franz Rottensteiner,et al.  ISPRS Test Project on Urban Classification and 3D Building Reconstruction: Evaluation of Building Reconstruction Results , 2009 .

[18]  Li Yong,et al.  ADAPTIVE BUILDING EDGE DETECTION BY COMBINING LIDAR DATA AND AERIAL IMAGES , 2008 .

[19]  Theodosios Pavlidis,et al.  Use of Shadows for Extracting Buildings in Aerial Images , 1990, Comput. Vis. Graph. Image Process..