Robust Domain-Free Domain Generalization with Class-Aware Alignment

While deep neural networks demonstrate state-of-the-art performance on a variety of learning tasks, their performance relies on the assumption that train and test distributions are the same, which may not hold in real-world applications. Domain generalization addresses this issue by employing multiple source domains to build robust models that can generalize to unseen target domains subject to shifts in data distribution. In this paper, we propose DomainFree Domain Generalization (DFDG), a model-agnostic method to achieve better generalization performance on the unseen test domain without the need for source domain labels. DFDG uses novel strategies to learn domain-invariant class-discriminative features. It aligns class relationships of samples through class-conditional soft labels, and uses saliency maps, traditionally developed for post-hoc analysis of image classification networks, to remove superficial observations from training inputs. DFDG obtains competitive performance on both time series sensor and image classification public datasets.

[1]  Robert B. Randall,et al.  Rolling element bearing diagnostics using the Case Western Reserve University data: A benchmark study , 2015 .

[2]  Yi Yang,et al.  Contrastive Adaptation Network for Unsupervised Domain Adaptation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Eric P. Xing,et al.  Learning Robust Representations by Projecting Superficial Statistics Out , 2018, ICLR.

[4]  Chih-Yao Ma,et al.  Frustratingly Simple Domain Generalization via Image Stylization , 2020, ArXiv.

[5]  Meng Chen,et al.  Intelligent Fault Diagnosis of Rotary Machinery Based on Unsupervised Multiscale Representation Learning , 2017 .

[6]  Wei Zhang,et al.  A New Deep Learning Model for Fault Diagnosis with Good Anti-Noise and Domain Adaptation Ability on Raw Vibration Signals , 2017, Sensors.

[7]  Aaron C. Courville,et al.  Out-of-Distribution Generalization via Risk Extrapolation (REx) , 2020, ICML.

[8]  Daniel C. Castro,et al.  Domain Generalization via Model-Agnostic Learning of Semantic Features , 2019, NeurIPS.

[9]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[10]  Tatsuya Harada,et al.  Domain Generalization Using a Mixture of Multiple Latent Domains , 2019, AAAI.

[11]  Ivan Marsic,et al.  Region-based Activity Recognition Using Conditional GAN , 2017, ACM Multimedia.

[12]  Silvio Savarese,et al.  Generalizing to Unseen Domains via Adversarial Data Augmentation , 2018, NeurIPS.

[13]  Eric P. Xing,et al.  Self-Challenging Improves Cross-Domain Generalization , 2020, ECCV.

[14]  Florian Lingenfelser,et al.  Relevance-Based Feature Masking: Improving Neural Network Based Whale Classification Through Explainable Artificial Intelligence , 2019, INTERSPEECH.

[15]  Minqiang Xu,et al.  Intelligent Fault Identification Based on Multisource Domain Generalization Towards Actual Diagnosis Scenario , 2020, IEEE Transactions on Industrial Electronics.

[16]  Rob J. Hyndman,et al.  Meta‐learning how to forecast time series , 2023, Journal of Forecasting.

[17]  Ioannis Mitliagkas,et al.  Generalizing to unseen domains via distribution matching , 2019 .

[18]  Yongxin Yang,et al.  Episodic Training for Domain Generalization , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Mikkel Baun Kjærgaard,et al.  Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition , 2015, SenSys.

[20]  Diane J. Cook,et al.  Multi-Source Deep Domain Adaptation with Weak Supervision for Time-Series Sensor Data , 2020, KDD.

[21]  Cheng-Lin Liu,et al.  Adaptive spatial pooling for image classification , 2016, Pattern Recognit..

[22]  Alex ChiChung Kot,et al.  Domain Generalization with Adversarial Feature Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Bogdan Gabrys,et al.  Meta-learning for time series forecasting and forecast combination , 2010, Neurocomputing.

[24]  Fabio Maria Carlucci,et al.  Domain Generalization by Solving Jigsaw Puzzles , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Stefano Soatto,et al.  Unsupervised Domain Adaptation via Regularized Conditional Alignment , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Donald A. Adjeroh,et al.  Unified Deep Supervised Domain Adaptation and Generalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Barbara Caputo,et al.  Best Sources Forward: Domain Generalization through Source-Specific Nets , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[28]  Yongxin Yang,et al.  Learning to Generalize: Meta-Learning for Domain Generalization , 2017, AAAI.

[29]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[30]  Emily Mower Provost,et al.  Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG) , 2019, IEEE Transactions on Affective Computing.

[31]  Barbara Caputo,et al.  Domain Generalization with Domain-Specific Aggregation Modules , 2018, GCPR.

[32]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Swami Sankaranarayanan,et al.  MetaReg: Towards Domain Generalization using Meta-Regularization , 2018, NeurIPS.

[34]  Martin Wattenberg,et al.  SmoothGrad: removing noise by adding noise , 2017, ArXiv.