LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

Deep learning approaches have shown promising results in remote sensing high 1 spatial resolution (HSR) land-cover mapping. However, urban and rural scenes 2 can show completely different geographical landscapes, and the inadequate gener3 alizability of these algorithms hinders city-level or national-level mapping. Most 4 of the existing HSR land-cover datasets only focus on improvement of the se5 mantic segmentation in one domain (urban or rural), thereby ignoring the model 6 transferability. In this paper, we introduce the Land-cOVEr Domain Adaptation 7 semantic segmentation (LoveDA) dataset to promote large-scale land-cover map8 ping. The LoveDA dataset contains 3338 aerial images with 86516 annotated 9 objects for seven common land-cover categories. Compared to the existing datasets, 10 the LoveDA dataset encompasses two domains (urban and rural), which brings 11 considerable challenges due to the: 1) multi-scale objects; 2) complex background 12 samples; and 3) inconsistent class distributions. The LoveDA dataset is suitable 13 for both land-cover semantic segmentation and unsupervised domain adaptation 14 (UDA) tasks. Accordingly, we benchmarked the LoveDA dataset on nine semantic 15 segmentation methods and eight UDA methods. Some exploratory studies were 16 also carried out to find alternative ways to address these challenges. The code and 17 data will be available at: https://github.com/Junjue-Wang/LoveDA 18

[1]  Pierre Alliez,et al.  StandardGAN: Multi-source Domain Adaptation for Semantic Segmentation of Very High Resolution Satellite Images by Data Standardization , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[3]  Kaiming He,et al.  Panoptic Feature Pyramid Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Kate Saenko,et al.  Deep CORAL: Correlation Alignment for Deep Domain Adaptation , 2016, ECCV Workshops.

[5]  Jing Huang,et al.  DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Shanghang Zhang,et al.  Instance Adaptive Self-Training for Unsupervised Domain Adaptation , 2020, ECCV.

[8]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[9]  Michele Volpi,et al.  Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[10]  Naif Alajlan,et al.  Domain Adaptation Network for Cross-Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Yi Yang,et al.  Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Wei Zhang,et al.  Classes Matter: A Fine-grained Adversarial Approach to Cross-domain Semantic Segmentation , 2020, ECCV.

[13]  Yang Zhao,et al.  Deep High-Resolution Representation Learning for Visual Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Michael Kampffmeyer,et al.  Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Hong Huang,et al.  Local Manifold-Based Sparse Discriminant Learning for Feature Extraction of Hyperspectral Image , 2020, IEEE Transactions on Cybernetics.

[16]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Kate Saenko,et al.  VisDA: A Synthetic-to-Real Benchmark for Visual Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Michael I. Jordan,et al.  Transferable Normalization: Towards Improving Transferability of Deep Neural Networks , 2019, NeurIPS.

[19]  Fengmao Lv,et al.  Constructing Self-Motivated Pyramid Curriculums for Cross-Domain Semantic Segmentation: A Non-Adversarial Approach , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[20]  Dominik Batorski,et al.  LandCover.ai: Dataset for Automatic Mapping of Buildings, Woodlands and Water from Aerial Imagery , 2020, ArXiv.

[21]  Bo Wang,et al.  Moment Matching for Multi-Source Domain Adaptation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Hamed Alemohammad,et al.  LandCoverNet: A global benchmark land cover classification training dataset , 2020, ArXiv.

[23]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[24]  Xiao Xiang Zhu,et al.  A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Qiangqiang Yuan,et al.  Satellite Video Super-Resolution via Multiscale Deformable Convolution Alignment and Temporal Grouping Projection , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Xiaofeng Liu,et al.  Confidence Regularized Self-Training , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[27]  Ling Shao,et al.  iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images , 2019, CVPR Workshops.

[28]  Chunhong Pan,et al.  Triplet Adversarial Domain Adaptation for Pixel-Level Classification of VHR Remote Sensing Images , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[29]  P. Gamba,et al.  Incorporating synthetic aperture radar and optical images to investigate the annual dynamics of anthropogenic impervious surface at large scale , 2020 .

[30]  Chao Tian,et al.  Dense Fusion Classmate Network for Land Cover Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[32]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[33]  Bo Du,et al.  Spectral–Spatial Weighted Kernel Manifold Embedded Distribution Alignment for Remote Sensing Image Classification , 2020, IEEE Transactions on Cybernetics.

[34]  Michele Volpi,et al.  Semantic segmentation of urban scenes by learning local class interactions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[35]  Ming-Hsuan Yang,et al.  Learning to Adapt Structured Output Space for Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Adam Van Etten,et al.  SpaceNet: A Remote Sensing Dataset and Challenge Series , 2018, ArXiv.

[37]  Dacheng Tao,et al.  Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation , 2019, NeurIPS.

[38]  Pengfei Xiong,et al.  Pyramid Attention Network for Semantic Segmentation , 2018, BMVC.

[39]  Zhuo Zheng,et al.  Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Trevor Darrell,et al.  Deep Domain Confusion: Maximizing for Domain Invariance , 2014, CVPR 2014.

[41]  Yifan Wu,et al.  Aerial Imagery for Roof Segmentation: A Large-Scale Dataset towards Automatic Mapping of Buildings , 2018, ArXiv.

[42]  Gui-Song Xia,et al.  Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Models , 2018 .

[43]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[44]  Xiangtao Zheng,et al.  Multisource Compensation Network for Remote Sensing Cross-Domain Scene Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Yang Zou,et al.  Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training , 2018, ArXiv.

[46]  J. Dewitz,et al.  National Land Cover Database (NLCD) 2016 Science Research Products , 2019 .

[47]  Zhuo Zheng,et al.  RSNet: The Search for Remote Sensing Deep Neural Networks in Recognition Tasks , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Mohsen Ali,et al.  Weakly Supervised Domain Adaptation for Built-up Region Segmentation in Aerial and Satellite Imagery , 2020, ArXiv.

[50]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[51]  Zhuo Zheng,et al.  FactSeg: Foreground Activation-Driven Small Object Semantic Segmentation in Large-Scale Remote Sensing Imagery , 2021, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Xiaoning Qian,et al.  Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  W. Tobler.  A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[54]  Sethuraman Panchanathan,et al.  Deep Hashing Network for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Eugenio Culurciello,et al.  LinkNet: Exploiting encoder representations for efficient semantic segmentation , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).