ESPC_NASUnet: An End-to-End Super-Resolution Semantic Segmentation Network for Mapping Buildings From Remote Sensing Images

Higher resolution building mapping from lower resolution remote sensing images is in great demand due to the lack of higher resolution data access, especially in the context of disaster assessment. High resolution building layout map is crucial for emergency rescue after the disaster. The emergency response time would be reduced if detailed building footprints were delineated from more easily available low-resolution data. To achieve this goal, we propose a super-resolution semantic segmentation network called ESPC_NASUnet, which consists of a feature super-resolution module and a semantic segmentation module. To the best of authors’ knowledge, this is the first work to systematically explore a deep learning-based approach to generate semantic maps with higher spatial resolution from lower spatial resolution remote sensing images in an end-to-end fashion. The experimental results for two datasets suggest that the proposed network is the best among four different end-to-end architectures in terms of both pixel-level metrics and object-level metrics. In terms of pixel-level $F$1-score, the improvements are greater than 0.068 and 0.055. Regarding the object-level $F$1-score, the disparities between ESPC_NASUnet and other end-to-end methods are more than 0.083 and 0.161 in the two datasets, respectively. Compared with stage-wise methods, our end-to-end network is less impacted by low-resolution input images. Finally, the proposed network produces building semantic maps comparable to those generated by semantic segmentation networks trained with high-resolution images and the ground truth utilizing the two datasets.