Deep Open-Set Domain Adaptation for Cross-Scene Classification based on Adversarial Learning and Pareto Ranking

Most of the existing domain adaptation (DA) methods proposed in the context of remote sensing imagery assume the presence of the same land-cover classes in the source and target domains. Yet, this assumption is not always realistic in practice as the target domain may contain additional classes unknown to the source leading to the so-called open set DA. Under this challenging setting, the problem turns to reducing the distribution discrepancy between the shared classes in both domains besides the detection of the unknown class samples in the target domain. To deal with the openset problem, we propose an approach based on adversarial learning and pareto-based ranking. In particular, the method leverages the distribution discrepancy between the source and target domains using min-max entropy optimization. During the alignment process, it identifies candidate samples of the unknown class from the target domain through a pareto-based ranking scheme that uses ambiguity criteria based on entropy and the distance to source class prototype. Promising results using two cross-domain datasets that consist of very high resolution and extremely high resolution images, show the effectiveness of the proposed method.

[1]  Xuelong Li,et al.  Scene Classification With Recurrent Attention of VHR Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Wei-Shi Zheng,et al.  Weakly Supervised Open-Set Domain Adaptation by Dual-Domain Collaboration , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Xiaodong Mu,et al.  Remote sensing image scene classification based on generative adversarial networks , 2018 .

[4]  Jing Zhang,et al.  Importance Weighted Adversarial Nets for Partial Domain Adaptation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Tatsuya Harada,et al.  Open Set Domain Adaptation by Backpropagation , 2018, ECCV.

[6]  Terrance E. Boult,et al.  Towards Open Set Deep Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Sergio Escalera,et al.  Recurrent neural networks for remote sensing image classification , 2018, IET Comput. Vis..

[9]  Jefersson Alex dos Santos,et al.  Improving Spatial Feature Representation from Aerial Scenes by Using Convolutional Networks , 2015, 2015 28th SIBGRAPI Conference on Graphics, Patterns and Images.

[10]  Rahil Garnavi,et al.  Generative OpenMax for Multi-Class Open Set Classification , 2017, BMVC.

[11]  Anil M. Cheriyadat,et al.  Unsupervised Feature Learning for Aerial Scene Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Edmund G. Zelnio,et al.  Open set SAR target classification , 2019, Defense + Commercial Sensing.

[13]  Naif Alajlan,et al.  Siamese-GAN: Learning Invariant Representations for Aerial Vehicle Image Categorization , 2018, Remote. Sens..

[14]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[15]  Naif Alajlan,et al.  Land-Use Classification With Compressive Sensing Multifeature Fusion , 2015, IEEE Geoscience and Remote Sensing Letters.

[16]  Terrance E. Boult,et al.  Towards Open World Recognition , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Naif Alajlan,et al.  Using convolutional features and a sparse autoencoder for land-use scene classification , 2016 .

[20]  Yakoub Bazi,et al.  Asymmetric Adaptation of Deep Features for Cross-Domain Classification in Remote Sensing Imagery , 2018, IEEE Geoscience and Remote Sensing Letters.

[21]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[22]  Bo Du,et al.  Domain Adaptation With Discriminative Distribution and Manifold Embedding for Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[23]  Liangpei Zhang,et al.  A Deep-Local-Global Feature Fusion Framework for High Spatial Resolution Imagery Scene Classification , 2018, Remote. Sens..

[24]  Lei Shu,et al.  DOC: Deep Open Classification of Text Documents , 2017, EMNLP.

[25]  Bo Du,et al.  Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the Art , 2016, IEEE Geoscience and Remote Sensing Magazine.

[26]  Ye Li,et al.  Hybrid Collaborative Representation for Remote-Sensing Image Scene Classification , 2018, Remote. Sens..

[27]  Terrance E. Boult,et al.  Multi-class Open Set Recognition Using Probability of Inclusion , 2014, ECCV.

[28]  Uwe Stilla,et al.  Deep Learning Earth Observation Classification Using ImageNet Pretrained Networks , 2016, IEEE Geoscience and Remote Sensing Letters.

[29]  Jianmin Wang,et al.  Partial Transfer Learning with Selective Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Qian Du,et al.  Remote Sensing Image Scene Classification Using Multi-Scale Completed Local Binary Patterns and Fisher Vectors , 2016, Remote. Sens..

[31]  Weng-Keen Wong,et al.  Open Set Learning with Counterfactual Images , 2018, ECCV.

[32]  Naif Alajlan,et al.  Domain Adaptation Network for Cross-Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[33]  Juergen Gall,et al.  Open Set Domain Adaptation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  Brian D. Rigling,et al.  Open set recognition for automatic target classification with rejection , 2016, IEEE Transactions on Aerospace and Electronic Systems.

[35]  Dan Zeng,et al.  Improving Remote Sensing Scene Classification by Integrating Global-Context and Local-Object Features , 2018, Remote. Sens..

[36]  Hong Liu,et al.  Separate to Adapt: Open Set Domain Adaptation via Progressive Separation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Yang Li,et al.  Open Set Radar HRRP Recognition Based on Random Forest and Extreme Value Theory , 2018, 2018 International Conference on Radar (RADAR).

[38]  Vishal M. Patel,et al.  Sparse Representation-Based Open Set Recognition , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Anderson Rocha,et al.  Toward Open Set Recognition , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Yuan Yan Tang,et al.  Dictionary Learning-Based Feature-Level Domain Adaptation for Cross-Scene Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.