Multisource Compensation Network for Remote Sensing Cross-Domain Scene Classification

Cross-domain scene classification refers to the scene classification task in which the training set (termed source domain) and the test set (termed target domain) come from different distributions. Various domain adaptation methods have been developed to reduce the distribution discrepancy between different domains. However, current domain adaptation methods assume that the source domain and target domain share the same categories. In reality, it is hard to find a source domain that can completely cover all the categories of target domain. In this article, we propose to use multiple complementary source domains to form the categories of target domain. A multisource compensation network (MSCN) is proposed to tackle these challenges: distribution discrepancy and category incompleteness. First, a pretrained convolutional neural network (CNN) is exploited to learn the feature representation for each domain. Second, a cross-domain alignment module is developed to reduce the domain shift between source and target domains. Domain shift is reduced by mapping the two domain features into a common feature space. Finally, a classifier complement module is proposed to align categories in multiple sources and learn a target classifier. Two cross-domain classification data sets are constructed using four heterogeneous remote sensing scene classification data sets. Extensive experiments are conducted on these datasets to validate the effectiveness of the proposed method. The proposed method can achieve 81.23% and 81.97% average accuracies on two-source-complementary data set and three-source-complementary data set, respectively.

[1]  Luisa Verdoliva,et al.  Land Use Classification in Remote Sensing Images by Convolutional Neural Networks , 2015, ArXiv.

[2]  José M. F. Moura,et al.  Adversarial Multiple Source Domain Adaptation , 2018, NeurIPS.

[3]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[4]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Michael I. Jordan,et al.  Conditional Adversarial Domain Adaptation , 2017, NeurIPS.

[6]  Xiangtao Zheng,et al.  A Deep Scene Representation for Aerial Scene Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Rong Yan,et al.  Cross-domain video concept detection using adaptive svms , 2007, ACM Multimedia.

[8]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[9]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[10]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[11]  Xiangtao Zheng,et al.  Semantic Descriptions of High-Resolution Remote Sensing Images , 2019, IEEE Geoscience and Remote Sensing Letters.

[12]  Naif Alajlan,et al.  Siamese-GAN: Learning Invariant Representations for Aerial Vehicle Image Categorization , 2018, Remote. Sens..

[13]  Ming Shao,et al.  Structure-Preserved Multi-source Domain Adaptation , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[14]  Xiangtao Zheng,et al.  Remote Sensing Scene Classification by Unsupervised Representation Learning , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Qiang Zhang,et al.  Domain Adaptation for Convolutional Neural Networks-Based Remote Sensing Scene Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[19]  Donald A. Adjeroh,et al.  Unified Deep Supervised Domain Adaptation and Generalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[21]  Zhenfeng Shao,et al.  PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[22]  Brian C. Lovell,et al.  Unsupervised Domain Adaptation by Domain Invariant Projection , 2013, 2013 IEEE International Conference on Computer Vision.

[23]  Wen Li,et al.  Domain Generalization and Adaptation Using Low Rank Exemplar SVMs , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Naif Alajlan,et al.  Domain Adaptation Network for Cross-Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Mohamed Lamine Mekhalfi,et al.  Learning a Multi-Branch Neural Network from Multiple Sources for Knowledge Adaptation in Remote Sensing Imagery , 2018, Remote. Sens..

[27]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Yakoub Bazi,et al.  Asymmetric Adaptation of Deep Features for Cross-Domain Classification in Remote Sensing Imagery , 2018, IEEE Geoscience and Remote Sensing Letters.

[29]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[30]  Xue Li,et al.  On Gleaning Knowledge From Cross Domains by Sparse Subspace Correlation Analysis for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Xiangtao Zheng,et al.  Hyperspectral Image Denoising by Fusing the Selected Related Bands , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Juergen Gall,et al.  Open Set Domain Adaptation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Li Yan,et al.  Cross-Domain Distance Metric Learning Framework With Limited Target Samples for Scene Classification of Aerial Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Fahad Shahbaz Khan,et al.  Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification , 2017, ArXiv.

[35]  Xuelong Li,et al.  Scene Classification With Recurrent Attention of VHR Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[37]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[38]  Xiangtao Zheng,et al.  Joint Dictionary Learning for Multispectral Change Detection , 2017, IEEE Transactions on Cybernetics.

[39]  Ming Yang,et al.  Conditional Generative Adversarial Network for Structured Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[40]  Andrea Vedaldi,et al.  Learning multiple visual domains with residual adapters , 2017, NIPS.

[41]  Ming Shao,et al.  Incomplete Multisource Transfer Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Yu Zhang,et al.  Transfer Learning via Learning to Transfer , 2018, ICML.

[43]  Liang Lin,et al.  Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Jian Shen,et al.  Wasserstein Distance Guided Representation Learning for Domain Adaptation , 2017, AAAI.

[45]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification Using Rearranged Local Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[47]  Michael I. Jordan,et al.  Deep Transfer Learning with Joint Adaptation Networks , 2016, ICML.

[48]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[49]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[50]  Trevor Darrell,et al.  Simultaneous Deep Transfer Across Domains and Tasks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).