WSF-NET: Weakly Supervised Feature-Fusion Network for Binary Segmentation in Remote Sensing Image

Binary segmentation in remote sensing aims to obtain binary prediction mask classifying each pixel in the given image. Deep learning methods have shown outstanding performance in this task. These existing methods in fully supervised manner need massive high-quality datasets with manual pixel-level annotations. However, the annotations are generally expensive and sometimes unreliable. Recently, using only image-level annotations, weakly supervised methods have proven to be effective in natural imagery, which significantly reduce the dependence on manual fine labeling. In this paper, we review existing methods and propose a novel weakly supervised binary segmentation framework, which is capable of addressing the issue of class imbalance via a balanced binary training strategy. Besides, a weakly supervised feature-fusion network (WSF-Net) is introduced to adapt to the unique characteristics of objects in remote sensing image. The experiments were implemented on two challenging remote sensing datasets: Water dataset and Cloud dataset. Water dataset is acquired by Google Earth with a resolution of 0.5 m, and Cloud dataset is acquired by Gaofen-1 satellite with a resolution of 16 m. The results demonstrate that using only image-level annotations, our method can achieve comparable results to fully supervised methods.

[1]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[2]  Menglong Yan,et al.  Change Detection Based on Deep Siamese Convolutional Network for Optical Aerial Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[3]  Trevor Darrell,et al.  Constrained Convolutional Neural Networks for Weakly Supervised Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Bolei Zhou,et al.  Object Detectors Emerge in Deep Scene CNNs , 2014, ICLR.

[5]  Hongliang Li,et al.  Semantic Annotation of Satellite Images Using Author–Genre–Topic Model , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Jian Yang,et al.  A Modified Level Set Approach for Segmentation of Multiband Polarimetric SAR Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Christoph H. Lampert,et al.  Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation , 2016, ECCV.

[8]  Thomas Hofmann,et al.  Support Vector Machines for Multiple-Instance Learning , 2002, NIPS.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  He Chen,et al.  Harbor Water Area Extraction From Pan-Sharpened Remotely Sensed Images Based on the Definition Circle Model , 2017, IEEE Geoscience and Remote Sensing Letters.

[12]  Yunchao Wei,et al.  Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Ali Ozgun Ok,et al.  Automated detection of buildings from single VHR multispectral images using shadow information and graph cuts , 2013 .

[14]  Xiaopeng Zhang,et al.  Robust Rooftop Extraction From Visible Band Images Using Higher Order CRF , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Menglong Yan,et al.  Automatic Water-Body Segmentation From High-Resolution Satellite Images via Deep Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[16]  Zhenwei Shi,et al.  Maritime Semantic Labeling of Optical Remote Sensing Images with Multi-Scale Fully Convolutional Network , 2017, Remote. Sens..

[17]  Margarida Silveira,et al.  Separation Between Water and Land in SAR Images Using Region-Based Level Sets , 2009, IEEE Geoscience and Remote Sensing Letters.

[18]  Menglong Yan,et al.  Semantic Segmentation of Aerial Images With Shuffling Convolutional Neural Networks , 2018, IEEE Geoscience and Remote Sensing Letters.

[19]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[21]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[22]  Satoshi Tsutsui,et al.  Distantly Supervised Road Segmentation , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[23]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Jian Sun,et al.  BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[27]  Ying Wang,et al.  Gated Convolutional Neural Network for Semantic Segmentation in High-Resolution Images , 2017, Remote. Sens..

[28]  Matthieu Cord,et al.  WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Yuri Boykov,et al.  Normalized Cut Loss for Weakly-Supervised CNN Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  N. Kanopoulos,et al.  Design of an image edge detection filter using the Sobel operator , 1988, IEEE J. Solid State Circuits.

[31]  Elsa D. Angelini,et al.  Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules , 2017, MICCAI.

[32]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[33]  Yiquan Wu,et al.  A new active contour remote sensing river image segmentation algorithm inspired from the cross entropy , 2016, Digit. Signal Process..

[34]  Bernt Schiele,et al.  Simple Does It: Weakly Supervised Instance and Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Yongyang Xu,et al.  Building Extraction in Very High Resolution Remote Sensing Imagery Using Deep Learning and Guided Filters , 2018, Remote. Sens..

[36]  Marcin Ciecholewski,et al.  River channel segmentation in polarimetric SAR images: Watershed transform combined with average contrast maximisation , 2017, Expert Syst. Appl..

[37]  Chris A. Glasbey,et al.  An Analysis of Histogram-Based Thresholding Algorithms , 1993, CVGIP Graph. Model. Image Process..

[38]  Gérard G. Medioni,et al.  Fast Convolution with Laplacian-of-Gaussian Masks , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Ronan Collobert,et al.  From image-level to pixel-level labeling with Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Menglong Yan,et al.  Semantic pixel labelling in remote sensing images using a deep convolutional encoder-decoder model , 2018 .