Boundary Loss for Remote Sensing Imagery Semantic Segmentation

In response to the growing importance of geospatial data, its analysis including semantic segmentation becomes an increasingly popular task in computer vision today. Convolutional neural networks are powerful visual models that yield hierarchies of features and practitioners widely use them to process remote sensing data. When performing remote sensing image segmentation, multiple instances of one class with precisely defined boundaries are often the case, and it is crucial to extract those boundaries accurately. The accuracy of segments boundaries delineation influences the quality of the whole segmented areas explicitly. However, widely-used segmentation loss functions such as BCE, IoU loss or Dice loss do not penalize misalignment of boundaries sufficiently. In this paper, we propose a novel loss function, namely a differentiable surrogate of a metric accounting accuracy of boundary detection. We can use the loss function with any neural network for binary segmentation. We performed validation of our loss function with various modifications of UNet on a synthetic dataset, as well as using real-world data (ISPRS Potsdam, INRIA AIL). Trained with the proposed loss function, models outperform baseline methods in terms of IoU score.

[1]  Pierre Alliez,et al.  Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[2]  Sebastian Nowozin,et al.  Optimal Decisions from Probabilistic Models: The Intersection-over-Union Case , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Pascal Fua,et al.  Beyond the Pixel-Wise Loss for Topology-Aware Delineation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Patrick Rives,et al.  A New Metric for Evaluating Semantic Segmentation: Leveraging Global and Contour Accuracy , 2017, 2018 IEEE Intelligent Vehicles Symposium (IV).

[5]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[7]  Markus Gerke,et al.  The ISPRS benchmark on urban object classification and 3D building reconstruction , 2012 .

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  Matthew B. Blaschko,et al.  The Lovasz-Softmax Loss: A Tractable Surrogate for the Optimization of the Intersection-Over-Union Measure in Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Jun Zhou,et al.  Conditional Random Field and Deep Feature Learning for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  A. Cichocki,et al.  Fast multispectral deep fusion networks , 2018 .

[14]  Bastian Leibe,et al.  Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Evgeny Burnaev,et al.  Targeted change detection in remote sensing images , 2019, International Conference on Machine Vision.

[16]  Gattigorla Nagendar,et al.  Neuro-IoU: Learning a Surrogate Loss for Semantic Segmentation , 2018, BMVC.

[17]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[18]  Evgeny Burnaev,et al.  Conformal k-NN Anomaly Detector for Univariate Data Streams , 2017, COPA.

[19]  Alexander Gammerman,et al.  Inductive Conformal Martingales for Change-Point Detection , 2017, COPA.

[20]  Evgeny Burnaev,et al.  Satellite imagery analysis for operational damage assessment in Emergency situations , 2018, BIS.

[21]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Evgeny Burnaev,et al.  Influence of resampling on accuracy of imbalanced classification , 2015, International Conference on Machine Vision.

[23]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[24]  Gabriela Csurka,et al.  What is a good evaluation measure for semantic segmentation? , 2013, BMVC.

[25]  Evgeny Burnaev,et al.  Meta-learning for resampling recommendation systems , 2017, International Conference on Machine Vision.

[26]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Evgeny Burnaev,et al.  Large-Scale Shape Retrieval with Sparse 3D Convolutional Neural Networks , 2017, AIST.

[28]  Ishfaq Ahmad,et al.  Superpixel-Enhanced Pairwise Conditional Random Field for Semantic Segmentation , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[29]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.