Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference

In this work, we propose a domain generalization (DG) approach to learn on several labeled source domains and transfer knowledge to a target domain that is inaccessible in training. Considering the inherent conditional and label shifts, we would expect the alignment of p(x|y) and p(y). However, the widely used domain invariant feature learning (IFL) methods relies on aligning the marginal concept shift w.r.t. p(x), which rests on an unrealistic assumption that p(y) is invariant across domains. We thereby propose a novel variational Bayesian inference framework to enforce the conditional distribution alignment w.r.t. p(x|y) via the prior distribution matching in a latent space, which also takes the marginal label shift w.r.t. p(y) into consideration with the posterior alignment. Extensive experiments on various benchmarks demonstrate that our framework is robust to the label shift and the crossdomain accuracy is significantly improved, thereby achieving superior performance over the conventional IFL counterparts.

[1]  D. Tao,et al.  Deep Domain Generalization via Conditional Invariant Adversarial Networks , 2018, ECCV.

[2]  Mengjie Zhang,et al.  Scatter Component Analysis: A Unified Framework for Domain Adaptation and Domain Generalization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Yongxin Yang,et al.  Deeper, Broader and Artier Domain Generalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Bernhard Schölkopf,et al.  On causal and anticausal learning , 2012, ICML.

[5]  Ling Shao,et al.  Learning to Learn with Variational Information Bottleneck for Domain Generalization , 2020, ECCV.

[6]  Tatsuya Harada,et al.  Domain Generalization Using a Mixture of Multiple Latent Domains , 2019, AAAI.

[7]  Zhitang Chen,et al.  Domain Generalization via Multidomain Discriminant Analysis , 2019, UAI.

[8]  Dacheng Tao,et al.  Causal Generative Domain Adaptation Networks , 2018, ArXiv.

[9]  Wouter M. Kouw An introduction to domain adaptation and transfer learning , 2018, ArXiv.

[10]  Rongzhong Lian,et al.  Learning to Select Knowledge for Response Generation in Dialog Systems , 2019, IJCAI.

[11]  Yongxin Yang,et al.  Episodic Training for Domain Generalization , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Geoffrey J. Gordon,et al.  Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift , 2020, NeurIPS.

[13]  Lequan Yu,et al.  Learning from Extrinsic and Intrinsic Supervisions for Domain Generalization , 2020, ECCV.

[14]  Yun Fu,et al.  Deep Domain Generalization With Structured Low-Rank Constraint , 2018, IEEE Transactions on Image Processing.

[15]  Siddhartha Chaudhuri,et al.  Generalizing Across Domains via Cross-Gradient Training , 2018, ICLR.

[16]  Alex ChiChung Kot,et al.  Domain Generalization with Adversarial Feature Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Bernhard Schölkopf,et al.  Domain Adaptation under Target and Conditional Shift , 2013, ICML.

[18]  Bernhard Schölkopf,et al.  Domain Adaptation with Conditional Transferable Components , 2016, ICML.

[19]  Bernhard Schölkopf,et al.  Multi-Source Domain Adaptation: A Causal View , 2015, AAAI.

[20]  C.-C. Jay Kuo,et al.  Mutual Information Regularized Feature-level Frankenstein for Discriminative Recognition. , 2021, IEEE transactions on pattern analysis and machine intelligence.

[21]  Xiaofeng Liu,et al.  Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosis , 2021, AAAI.

[22]  Justin Domke,et al.  Importance Weighting and Variational Inference , 2018, NeurIPS.

[23]  Eric P. Xing,et al.  Self-Challenging Improves Cross-Domain Generalization , 2020, ECCV.

[24]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[25]  Francisco Herrera,et al.  A unifying view on dataset shift in classification , 2012, Pattern Recognit..

[26]  Donald A. Adjeroh,et al.  Unified Deep Supervised Domain Adaptation and Generalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Xiaofeng Liu,et al.  Deep Verifier Networks: Verification of Deep Discriminative Models with Deep Generative Models , 2019, AAAI.

[28]  Barbara Caputo,et al.  Best Sources Forward: Domain Generalization through Source-Specific Nets , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[29]  Bianca Zadrozny,et al.  Learning and evaluating classifiers under sample selection bias , 2004, ICML.

[30]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.