SCIDA: Self-Correction Integrated Domain Adaptation From Single- to Multi-Label Aerial Images

Most publicly available datasets for image classification are with single labels, while images are inherently multilabeled in our daily life. Such an annotation gap makes many pre-trained single-label classification models fail in practical scenarios. This annotation issue is more concerned for aerial images: Aerial data collected from sensors naturally cover a relatively large land area with multiple labels, while annotated aerial datasets, which are publicly available (e.g., UCM, AID), are single-labeled. As manually annotating multi-label aerial images would be time/labor-consuming, we propose a novel selfcorrection integrated domain adaptation (SCIDA) method for automatic multi-label learning. SCIDA is weakly supervised, i.e., automatically learning the multi-label image classification model from using massive, publicly available single-label images. To achieve this goal, we propose a novel Label-Wise self-Correction (LWC) module to better explore underlying label correlations. This module also makes the unsupervised domain adaptation (UDA) from singleto multi-label data possible. For model training, the proposed model only uses single-label information yet requires no prior knowledge of multi-labeled data; and it predicts labels for multi-label aerial images. In our experiments, trained with single-labeled MAI-AID-s and MAI-UCM-s datasets, the proposed model is tested directly on our collected Multi-scene Aerial Image (MAI) dataset. The code and data are available on GitHub(https://github.com/Ryan315/Single2multi-DA).

[1]  Masashi Sugiyama,et al.  Progressive Identification of True Labels for Partial-Label Learning , 2020, ICML.

[2]  Lihi Zelnik-Manor,et al.  Asymmetric Loss For Multi-Label Classification , 2020, ArXiv.

[3]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[4]  Carlos D. Castillo,et al.  Generate to Adapt: Aligning Domains Using Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Luo Si,et al.  Binary Codes Embedding for Fast Image Tagging with Incomplete Labels , 2014, ECCV.

[6]  Deqing Wang,et al.  Aligning Domain-Specific Distribution and Classifier for Cross-Domain Classification from Multiple Sources , 2019, AAAI.

[7]  Yu-Chiang Frank Wang,et al.  Deep Generative Models for Weakly-Supervised Multi-Label Classification , 2018, ECCV.

[8]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[9]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Min-Ling Zhang,et al.  Disambiguation-Free Partial Label Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[11]  Liang Lin,et al.  Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  George Trigeorgis,et al.  Domain Separation Networks , 2016, NIPS.

[13]  Allan Jabri,et al.  Learning Visual Features from Large Weakly Supervised Data , 2015, ECCV.

[14]  Jianmin Wang,et al.  Multi-Adversarial Domain Adaptation , 2018, AAAI.

[15]  Bo An,et al.  Leveraging Latent Label Distributions for Partial Label Learning , 2018, IJCAI.

[16]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jiaying Liu,et al.  Adaptive Batch Normalization for practical domain adaptation , 2018, Pattern Recognit..

[18]  Ross B. Girshick,et al.  Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Kaiming He,et al.  Exploring the Limits of Weakly Supervised Pretraining , 2018, ECCV.

[20]  Yidong Li,et al.  Partial Multi-Label Learning via Probabilistic Graph Matching Mechanism , 2020, KDD.

[21]  Chen-Yu Lee,et al.  Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Yi Liu,et al.  Semi-supervised Multi-label Learning by Constrained Non-negative Matrix Factorization , 2006, AAAI.

[23]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[24]  Yun Fu,et al.  Adaptive Graph Guided Embedding for Multi-label Annotation , 2018, IJCAI.

[25]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[26]  Greg Mori,et al.  Learning a Deep ConvNet for Multi-Label Classification With Partial Labels , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Junghoon Seo,et al.  On The Power of Deep But Naive Partial Label Learning , 2020, ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Fabio Maria Carlucci,et al.  AutoDIAL: Automatic Domain Alignment Layers , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Sheng-Jun Huang,et al.  Partial Multi-Label Learning With Noisy Label Identification , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Taesung Park,et al.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation , 2017, ICML.

[32]  Wei Xing,et al.  ARC: Adversarial Robust Cuts for Semi-Supervised and Multi-label Classification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[33]  Rong Jin,et al.  Multi-label learning with incomplete class assignments , 2011, CVPR 2011.

[34]  Dat T. Huynh,et al.  Interactive Multi-Label CNN Learning With Partial Labels , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Zhi-Hua Zhou,et al.  Learning From Semi-Supervised Weak-Label Data , 2018, AAAI.

[36]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[39]  Chen Sun,et al.  Revisiting Unreasonable Effectiveness of Data in Deep Learning Era , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[40]  Masashi Sugiyama,et al.  Unsupervised Domain Adaptation Based on Source-guided Discrepancy , 2018, AAAI.

[41]  Ashish Kapoor,et al.  Active learning for sparse bayesian multilabel classification , 2014, KDD.

[42]  Jiaying Liu,et al.  Revisiting Batch Normalization For Practical Domain Adaptation , 2016, ICLR.

[43]  Min-Ling Zhang,et al.  Partial Multi-Label Learning via Credible Label Elicitation , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Jinwen Ma,et al.  Multi-Label Classification with Label Graph Superimposing , 2019, AAAI.

[45]  Weiwei Liu,et al.  Discriminative and Correlative Partial Multi-Label Learning , 2019, IJCAI.

[46]  Tao Wang,et al.  Partial Multi-Label Learning by Low-Rank and Sparse Decomposition , 2019, AAAI.

[47]  Xiu-Shen Wei,et al.  Multi-Label Image Recognition With Graph Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Céline Hudelot,et al.  Learning More Universal Representations for Transfer-Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[50]  Michael S. Bernstein,et al.  Scalable multi-label annotation , 2014, CHI.

[51]  Jianmin Wang,et al.  Partial Transfer Learning with Selective Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.