Disentangling Observed Causal Effects from Latent Confounders using Method of Moments

Discovering the complete set of causal relations among a group of variables is a challenging unsupervised learning problem. Often, this challenge is compounded by the fact that there are latent or hidden confounders. When only observational data is available, the problem is ill-posed, i.e. the causal relationships are nonidentifiable unless strong modeling assumptions are made. When (soft) interventions are available, we provide guarantees on identifiability and learnability under mild assumptions. We assume a linear structural equation model (SEM) with independent latent factors and directed acyclic graph (DAG) relationships among the observables. Since the latent variable inference is based on independent component analysis (ICA), we call this model SEM-ICA. We use the method of moments principle to establish model identifiability. We develop efficient algorithms based on coupled tensor decomposition with linear constraints to obtain scalable and guaranteed solutions. Thus, we provide a principled approach to tackling the joint problem of causal discovery and latent variable inference.

[1]  Elias Bareinboim,et al.  Budgeted Experiment Design for Causal Structure Learning , 2017, ICML.

[2]  Maja Pantic,et al.  TensorLy: Tensor Learning in Python , 2016, J. Mach. Learn. Res..

[3]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[4]  Frederick Eberhardt,et al.  Learning linear cyclic causal models with latent variables , 2012, J. Mach. Learn. Res..

[5]  Aapo Hyvärinen,et al.  A Linear Non-Gaussian Acyclic Model for Causal Discovery , 2006, J. Mach. Learn. Res..

[6]  Jianqing Fan,et al.  An l∞ Eigenvector Perturbation Bound and Its Application to Robust Covariance Estimation , 2018, Journal of machine learning research : JMLR.

[7]  Zhitang Chen,et al.  Causality in Linear Nongaussian Acyclic Models in the Presence of Latent Gaussian Confounders , 2013, Neural Computation.

[8]  Constantinos Daskalakis,et al.  Learning and Testing Causal Models with Interventions , 2018, NeurIPS.

[9]  Frederick Eberhardt,et al.  Constraint-based Causal Discovery: Conflict Resolution with Answer Set Programming , 2014, UAI.

[10]  Donatello Materassi,et al.  An Algorithm to Learn Polytree Networks with Hidden Nodes , 2019, NeurIPS.

[11]  Mathias Frisch,et al.  Causation and intervention , 2014 .

[12]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[13]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[14]  Frederick Eberhardt,et al.  N-1 Experiments Suffice to Determine the Causal Relations Among N Variables , 2006 .

[15]  Aapo Hyvärinen,et al.  ParceLiNGAM: A Causal Ordering Method Robust Against Latent Confounders , 2013, Neural Computation.

[16]  Richard Scheines,et al.  Learning the Structure of Linear Latent Variable Models , 2006, J. Mach. Learn. Res..

[17]  AmirEmad Ghassami,et al.  Learning Linear Non-Gaussian Causal Models in the Presence of Latent Variables , 2019, J. Mach. Learn. Res..

[18]  Aapo Hyvärinen,et al.  On the Identifiability of the Post-Nonlinear Causal Model , 2009, UAI.

[19]  Jonas Peters,et al.  BACKSHIFT: Learning causal cyclic graphs from unknown shift interventions , 2015, NIPS.

[20]  AmirEmad Ghassami,et al.  Interventional Experiment Design for Causal Structure Learning , 2019, ArXiv.

[21]  Aapo Hyvärinen,et al.  Variational Autoencoders and Nonlinear ICA: A Unifying Framework , 2019, AISTATS.

[22]  David M. Blei,et al.  The Blessings of Multiple Causes , 2018, Journal of the American Statistical Association.

[23]  Anima Anandkumar,et al.  Learning Overcomplete Latent Variable Models through Tensor Methods , 2014, COLT.

[24]  Aapo Hyvärinen,et al.  DirectLiNGAM: A Direct Method for Learning a Linear Non-Gaussian Structural Equation Model , 2011, J. Mach. Learn. Res..

[25]  Andrew McGregor,et al.  Efficient Intervention Design for Causal Discovery with Latents , 2020, ICML.

[26]  Bernhard Schölkopf,et al.  Nonlinear causal discovery with additive noise models , 2008, NIPS.

[27]  Adel Javanmard,et al.  Learning Linear Bayesian Networks with Latent Variables , 2012, ICML.

[28]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[29]  Kevin P. Murphy,et al.  Exact Bayesian structure learning from uncertain interventions , 2007, AISTATS.

[30]  Soon-Jo Chung,et al.  Neural Lander: Stable Drone Landing Control Using Learned Dynamics , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[31]  R. Scheines,et al.  Interventions and Causal Inference , 2007, Philosophy of Science.

[32]  Jimeng Sun,et al.  Guaranteed Scalable Learning of Latent Tree Models , 2014, UAI.

[33]  Peter Bühlmann,et al.  Characterization and Greedy Learning of Interventional Markov Equivalence Classes of Directed Acyclic Graphs (Abstract) , 2011, UAI.

[34]  Wei Lu,et al.  Eliminating indeterminacy in ICA , 2003, Neurocomputing.

[35]  Yangbo He,et al.  Active Learning of Causal Networks with Intervention Experiments and Optimal Designs , 2008 .

[36]  Patrik O. Hoyer,et al.  Estimation of causal effects using linear non-Gaussian causal models with hidden variables , 2008, Int. J. Approx. Reason..

[37]  Karthikeyan Shanmugam,et al.  Experimental Design for Learning Causal Graphs with Latent Variables , 2017, NIPS.