3D Semantic Segmentation from Multi-View Optical Satellite Images

This paper describes the winning contribution to the 2019 IEEE GRSS Data Fusion Contest Multi-view Semantic Stereo Challenge. In this challenge, a digital surface model (DSM) and a semantic segmentation should be derived from a large number of multi-spectral WorldView-3 images. Results from 50 stereo pairs matched using Semi-Global Matching (SGM) are fused into a DSM. Semantic segmentation is performed with an ensemble of FCN networks taking as input RGB, multi-spectral and height data. Their results are then merged with pixel-wise detectors for the classes water and high vegetation. Compared to the second and third placed teams (mIOU-3 scores of 0.73 and 0.7295), our contribution reached a significantly higher score of 0.745.

[1]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Bastian Leibe,et al.  Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Gregory D. Hager,et al.  Semantic Stereo for Incidental Satellite Images , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Pablo d'Angelo,et al.  Dense multi-view stereo from satellite imagery , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[7]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Fred A. Kruse,et al.  The Spectral Image Processing System (SIPS) - Interactive visualization and analysis of imaging spectrometer data , 1993 .

[9]  Pablo d'Angelo.  Automatic Orientation of large multitemporal Satellite Image Blocks , 2013 .

[10]  Peter Reinartz,et al.  Aerial LaneNet: Lane-Marking Semantic Segmentation in Aerial Imagery Using Wavelet-Enhanced Cost-Sensitive Symmetric Fully Convolutional Neural Networks , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[11]  S. K. McFeeters.  The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features , 1996 .

[12]  Mathias Schardt,et al.  Advanced DTM Generation from Very High Resolution Satellite Stereo Images , 2015 .

[13]  Naoto Yokoya,et al.  2019 Data Fusion Contest [Technical Committees] , 2019, IEEE Geoscience and Remote Sensing Magazine.