Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions

An important goal common to domain adaptation and causal inference is to make accurate predictions when the distributions for the source (or training) domain(s) and target (or test) domain(s) differ. In many cases, these different distributions can be modeled as different contexts of a single underlying system, in which each distribution corresponds to a different perturbation of the system, or in causal terms, an intervention. We focus on a class of such causal domain adaptation problems, where data for one or more source domains are given, and the task is to predict the distribution of a certain target variable from measurements of other variables in one or more target domains. We propose an approach for solving these problems that exploits causal inference and does not rely on prior knowledge of the causal graph, the type of interventions or the intervention targets. We demonstrate our approach by evaluating a possible implementation on simulated and real world data.

[1]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[2]  Bernhard Schölkopf,et al.  Theoretical Aspects of Cyclic Structural Causal Models , 2016 .

[3]  Judea Pearl,et al.  Comment: Graphical Models, Causality and Intervention , 2016 .

[4]  Elias Bareinboim,et al.  Causal inference and the data-fusion problem , 2016, Proceedings of the National Academy of Sciences.

[5]  T. Richardson Markov Properties for Acyclic Directed Mixed Graphs , 2003 .

[6]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[7]  Neil D. Lawrence,et al.  Dataset Shift in Machine Learning , 2009 .

[8]  Joris M. Mooij,et al.  Cyclic Causal Discovery from Continuous Equilibrium Data , 2013, UAI.

[9]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[10]  Bernhard Schölkopf,et al.  Multi-Source Domain Adaptation: A Causal View , 2015, AAAI.

[11]  Jiji Zhang,et al.  On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias , 2008, Artif. Intell..

[12]  Matti Järvisalo,et al.  Learning Optimal Causal Graphs with Exact Search , 2018, PGM.

[13]  Joris M. Mooij,et al.  Joint Causal Inference from Multiple Contexts , 2016, J. Mach. Learn. Res..

[14]  Elias Bareinboim,et al.  A General Algorithm for Deciding Transportability of Experimental Results , 2013, ArXiv.

[15]  Joris M. Mooij,et al.  Constraint-based Causal Discovery for Non-Linear Structural Causal Models with Cycles and Latent Confounders , 2018, UAI.

[16]  Frederick Eberhardt,et al.  Constraint-based Causal Discovery: Conflict Resolution with Answer Set Programming , 2014, UAI.

[17]  Rainer Spang,et al.  Probabilistic Soft Interventions in Conditional Gaussian Networks , 2005, AISTATS.

[18]  Bernhard Schölkopf,et al.  Invariant Models for Causal Transfer Learning , 2015, J. Mach. Learn. Res..

[19]  Kevin P. Murphy,et al.  Exact Bayesian structure learning from uncertain interventions , 2007, AISTATS.

[20]  Frederick Eberhardt,et al.  Do-calculus when the True Graph Is Unknown , 2015, UAI.

[21]  Ioannis Tsamardinos,et al.  Constraint-based causal discovery from multiple interventions over overlapping variable sets , 2014, J. Mach. Learn. Res..

[22]  Amos Storkey,et al.  When Training and Test Sets are Different: Characterising Learning Transfer , 2013 .

[23]  Elias Bareinboim,et al.  Transportability of Causal and Statistical Relations: A Formal Approach , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[24]  Gregory F. Cooper,et al.  A Simple Constraint-Based Algorithm for Efficiently Mining Observational Databases for Causal Relationships , 1997, Data Mining and Knowledge Discovery.

[25]  Motoaki Kawanabe,et al.  Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , 2007, NIPS.

[26]  Neil D. Lawrence,et al.  When Training and Test Sets Are Different: Characterizing Learning Transfer , 2009 .

[27]  Jin Tian,et al.  Causal Discovery from Changes , 2001, UAI.

[28]  Bernhard Schölkopf,et al.  Domain Adaptation with Conditional Transferable Components , 2016, ICML.

[29]  Bernhard Schölkopf,et al.  Domain Adaptation under Target and Conditional Shift , 2013, ICML.

[30]  Yishay Mansour,et al.  Domain Adaptation: Learning Bounds and Algorithms , 2009, COLT.

[31]  Joris M. Mooij,et al.  Structural Causal Models: Cycles, Marginalizations, Exogenous Reparametrizations and Reductions , 2016, ArXiv.

[32]  Joris M. Mooij,et al.  Ancestral Causal Inference , 2016, NIPS.

[33]  Bernhard Schölkopf,et al.  On causal and anticausal learning , 2012, ICML.

[34]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.