Improving Semantic Segmentation of Roof Segments Using Large-Scale Datasets Derived from 3D City Models and High-Resolution Aerial Imagery

Advances in deep learning techniques for remote sensing as well as the increased availability of high-resolution data enable the extraction of more detailed information from aerial images. One promising task is the semantic segmentation of roof segments and their orientation. However, the lack of annotated data is a major barrier for deploying respective models on a large scale. Previous research demonstrated the viability of the deep learning approach for the task, but currently, published datasets are small-scale, manually labeled, and rare. Therefore, this paper extends the state of the art by presenting a novel method for the automated generation of large-scale datasets based on semantic 3D city models. Furthermore, we train a model on a dataset 50 times larger than existing datasets and achieve superior performance while applying it to a wider variety of buildings. We evaluate the approach by comparing networks trained on four dataset configurations, including an existing dataset and our novel large-scale dataset. The results show that the network performance measured as intersection over union can be increased from 0.60 for the existing dataset to 0.70 when the large-scale model is applied on the same region. The large-scale model performs superiorly even when applied to more diverse test samples, achieving 0.635. The novel approach contributes to solving the dataset bottleneck and consequently to improving semantic segmentation of roof segments. The resulting remotely sensed information is crucial for applications such as solar potential analysis or urban planning.

[1]  Xiao Xiang Zhu,et al.  SolarNet: A convolutional neural network-based framework for rooftop solar potential estimation from aerial imagery , 2023, Int. J. Appl. Earth Obs. Geoinformation.

[2]  M. Lienkamp,et al.  RID - Roof Information Dataset for Computer Vision-Based Photovoltaic Potential Assessment , 2022, Remote. Sens..

[3]  Myungjoo Kang,et al.  Boundary Enhancement Semantic Segmentation for Building Extraction From Remote Sensed Image , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Yicheng Jiang,et al.  Refocusing on SAR Ship Targets With Three-Dimensional Rotating Based on Complex-Valued Convolutional Gated Recurrent Unit , 2022, IEEE Geoscience and Remote Sensing Letters.

[5]  Qinchuan Xin,et al.  A Deep Learning-Based Framework for Automated Extraction of Building Footprint Polygons from Very High-Resolution Aerial Imagery , 2021, Remote. Sens..

[6]  Lianpeng Zhang,et al.  Building Outline Extraction Directly Using the U2-Net Semantic Segmentation Model from High-Resolution Aerial Images and a Comparison Study , 2021, Remote. Sens..

[7]  Toshihiro Kawase,et al.  Loss Weightings for Improving Imbalanced Brain Structure Segmentation Using Fully Convolutional Networks , 2021, Healthcare.

[8]  Jessica Granderson,et al.  Open Data and Deep Semantic Segmentation for Automated Extraction of Building Footprints , 2021, Remote. Sens..

[9]  Markus Lienkamp,et al.  Towards Scalable Economic Photovoltaic Potential Analysis Using Aerial Images and Deep Learning , 2021, Energies.

[10]  Anima Anandkumar,et al.  SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers , 2021, NeurIPS.

[11]  S. Ganguly,et al.  Semantic Segmentation of High Resolution Satellite Imagery using Generative Adversarial Networks with Progressive Growing , 2021 .

[12]  Xiaohui Yuan,et al.  A review of deep learning methods for semantic segmentation of remote sensing imagery , 2021, Expert Syst. Appl..

[13]  Teerayut Horanont,et al.  Deep Learning-Based Semantic Segmentation of Urban Features in Satellite Images: A Review and Meta-Analysis , 2021, Remote. Sens..

[14]  Xian Sun,et al.  PiCoCo: Pixelwise Contrast and Consistency Learning for Semisupervised Building Footprint Segmentation , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Jingwen Yan,et al.  E-D-Net: Automatic Building Extraction From High-Resolution Aerial Images With Boundary Information , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Martin Jägersand,et al.  U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection , 2020, Pattern Recognit..

[17]  Mingsheng Liao,et al.  Remote Sensing Image Semantic Segmentation Based on Edge Information Guidance , 2020, Remote. Sens..

[18]  Meng Lu,et al.  Toward Automatic Building Footprint Delineation From Aerial Images Using CNN and Regularization , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[19]  Zenghui Zhang,et al.  GIS-Supervised Building Extraction With Label Noise-Adaptive Fully Convolutional Neural Network , 2020, IEEE Geoscience and Remote Sensing Letters.

[20]  Peter Caccetta,et al.  ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[21]  Biswajeet Pradhan,et al.  Building Footprint Extraction from High Resolution Aerial Images Using Generative Adversarial Network (GAN) Architecture , 2020, IEEE Access.

[22]  Subhransu Maji,et al.  DeepRoof: A Data-driven Approach For Solar Potential Estimation Using Rooftop Imagery , 2019, KDD.

[23]  Weijia Li,et al.  Semantic Segmentation-Based Building Footprint Extraction Using Very High-Resolution Satellite Images and Multi-Source GIS Data , 2019, Remote. Sens..

[24]  Jiangye Yuan,et al.  Learning Building Extraction in Aerial Scenes with Convolutional Networks , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Andreas Donaubauer,et al.  3DCityDB - a 3D geodatabase solution for the management, analysis, and visualization of semantic 3D city models based on CityGML , 2018, Open Geospatial Data, Software and Standards.

[26]  Wolf Fichtner,et al.  Assessment of rooftop photovoltaic potentials at the urban level using publicly available geodata and image recognition techniques , 2017 .

[27]  Derek Anderson,et al.  Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community , 2017 .

[28]  Thomas Hofmann,et al.  Learning Aerial Image Segmentation From Online Maps , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[30]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[31]  L. Bergamasco,et al.  Scalable methodology for the photovoltaic solar energy potential assessment based on available roof surface area: application to Piedmont Region (Italy) , 2011 .

[32]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[33]  Martin Kada,et al.  3D BUILDING RECONSTRUCTION FROM LIDAR BASED ON A CELL DECOMPOSITION APPROACH , 2009 .

[34]  C. Ratti,et al.  Solar Radiation over the Urban Texture: LIDAR Data and Image Processing Techniques for Environmental Analysis at City Scale , 2008 .