Learning to Induce Causal Structure

The fundamental challenge in causal induction is to infer the underlying graph structure given observational and/or interventional data. Most existing causal induction algorithms operate by generating candidate graphs and evaluating them using either score-based methods (including continuous optimization) or independence tests. In our work, we instead treat the inference process as a black box and design a neural network architecture that learns the mapping from both observational and interventional data to graph structures via supervised training on synthetic graphs. The learned model generalizes to new synthetic graphs, is robust to train-test distribution shifts, and achieves state-of-the-art performance on naturalistic graphs for low sample complexity. performs the worst, which is to be expect, since intervention data is need for causal identifiability. The performance of our model improves as it observes more interventions, this suggest that our model is able to extract useful information from interventions in order to predict the causal structure.

[1]  Nan Rosemary Ke,et al.  Retrieval-Augmented Reinforcement Learning , 2022, ICML.

[2]  Cherepanov,et al.  Competition-level code generation with AlphaCode , 2022, Science.

[3]  F. Castelletti,et al.  BCDAG: An R package for Bayesian structure and Causal learning of Gaussian DAGs , 2022, ArXiv.

[4]  Sebastian Pineda Arango,et al.  Transformers Can Do Bayesian Inference , 2021, ICLR.

[5]  Taco Cohen,et al.  Efficient Neural Causal Discovery without Acyclicity Constraints , 2021, ICLR.

[6]  Rémi Le Priol,et al.  Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA , 2021, CLeaR.

[7]  Nan Rosemary Ke,et al.  Coordination Among Neural Modules Through a Shared Global Workspace , 2021, ICLR.

[8]  R. Zemel,et al.  Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data , 2020, CLeaR.

[9]  Rui Ding,et al.  ML4C: Seeing Causality Through Latent Vicinity , 2021, ArXiv.

[10]  Nan Rosemary Ke,et al.  Learning Neural Causal Models with Active Interventions , 2021, ArXiv.

[11]  Danilo Jimenez Rezende,et al.  Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning , 2021, NeurIPS Datasets and Benchmarks.

[12]  Aidan N. Gomez,et al.  Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning , 2021, NeurIPS.

[13]  Zhitang Chen,et al.  Ordering-Based Causal Discovery with Reinforcement Learning , 2021, IJCAI.

[14]  Andrew Zisserman,et al.  Perceiver: General Perception with Iterative Attention , 2021, ICML.

[15]  Yoshua Bengio,et al.  CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning , 2020, ICLR.

[16]  Nan Rosemary Ke,et al.  Neural Production Systems , 2021, Neural Information Processing Systems.

[17]  Zeb Kurth-Nelson,et al.  Alchemy: A structured task distribution for meta-reinforcement learning , 2021, ArXiv.

[18]  Martin Szummer,et al.  Amortized learning of neural causal representations , 2020, ArXiv.

[19]  Alexandre Lacoste,et al.  Differentiable Causal Discovery from Interventional Data , 2020, NeurIPS.

[20]  Juan L Gamella,et al.  Active Invariant Causal Prediction: Experiment Selection through Stability , 2020, NeurIPS.

[21]  Qi Xiao,et al.  Supervised Whole DAG Causal Discovery , 2020, ArXiv.

[22]  知秀 柴田 5分で分かる!? 有名論文ナナメ読み:Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding , 2020 .

[23]  Caroline Uhler,et al.  Permutation-Based Causal Structure Learning with Unknown Intervention Targets , 2019, UAI.

[24]  Zhitang Chen,et al.  Causal Discovery with Reinforcement Learning , 2019, ICLR.

[25]  Tristan Deleu,et al.  Gradient-Based Neural DAG Learning , 2019, ICLR.

[26]  Christopher Joseph Pal,et al.  A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms , 2019, ICLR.

[27]  Joris M. Mooij,et al.  Joint Causal Inference from Multiple Contexts , 2016, J. Mach. Learn. Res..

[28]  Junier B. Oliva,et al.  Flow Models for Arbitrary Conditional Likelihoods , 2019, ArXiv.

[29]  P. Spirtes,et al.  Review of Causal Discovery Methods Based on Graphical Models , 2019, Front. Genet..

[30]  Mo Yu,et al.  DAG-GNN: DAG Structure Learning with Graph Neural Networks , 2019, ICML.

[31]  Aapo Hyvärinen,et al.  Causal Discovery with General Non-Linear Relationships using Non-Linear ICA , 2019, UAI.

[32]  Dmitry Vetrov,et al.  Variational Autoencoder with Arbitrary Conditioning , 2018, ICLR.

[33]  David Lopez-Paz,et al.  SAM: Structural Agnostic Model, Causal Discovery and Penalized Adversarial Learning , 2018 .

[34]  Bernhard Schölkopf,et al.  Generalized Score Functions for Causal Discovery , 2018, KDD.

[35]  Mihaela van der Schaar,et al.  GAIN: Missing Data Imputation using Generative Adversarial Nets , 2018, ICML.

[36]  Yee Whye Teh,et al.  Causal Inference via Kernel Deviance Measures , 2018, NeurIPS.

[37]  Pradeep Ravikumar,et al.  DAGs with NO TEARS: Continuous Optimization for Structure Learning , 2018, NeurIPS.

[38]  I. Guyon,et al.  Causal Generative Neural Networks , 2017, 1711.08936.

[39]  Christina Heinze-Deml,et al.  Invariant Causal Prediction for Nonlinear Models , 2017, Journal of Causal Inference.

[40]  Bernhard Schölkopf,et al.  Invariant Models for Causal Transfer Learning , 2015, J. Mach. Learn. Res..

[41]  Yura N. Perov,et al.  A Universal Marginalizer for Amortized Inference in Generative Models , 2017, ArXiv.

[42]  I. Guyon,et al.  Explainable and Interpretable Models in Computer Vision and Machine Learning , 2017, The Springer Series on Challenges in Machine Learning.

[43]  Christina Heinze-Deml,et al.  Causal Structure Learning , 2017, 1706.09141.

[44]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[45]  Kun Zhang,et al.  Learning Causal Structures Using Regression Invariance , 2017, NIPS.

[46]  Kailash Budhathoki,et al.  Causal Inference by Stochastic Complexity , 2017, ArXiv.

[47]  Marloes H. Maathuis,et al.  Structure Learning in Graphical Modeling , 2016, 1606.02359.

[48]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Bernhard Schölkopf,et al.  Towards a Learning Theory of Causation , 2015, 1502.02398.

[50]  Jonas Peters,et al.  Causal inference by using invariant prediction: identification and confidence intervals , 2015, 1501.01332.

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  David Lopez-Paz,et al.  The Randomized Causation Coefficient , 2014, J. Mach. Learn. Res..

[53]  Eric Chojnacki,et al.  An Efficient Bayesian Network Structure Learning Algorithm in the Presence of Deterministic Relations , 2014, ECAI.

[54]  T. Koski,et al.  A Review of Bayesian Networks and Structure Learning , 2012 .

[55]  Bernhard Schölkopf,et al.  Identifiability of Causal Graphs using Functional Models , 2011, UAI.

[56]  Bernhard Schölkopf,et al.  Kernel-based Conditional Independence Test and Application in Causal Discovery , 2011, UAI.

[57]  Peter Bühlmann,et al.  Characterization and Greedy Learning of Interventional Markov Equivalence Classes of Directed Acyclic Graphs (Abstract) , 2011, UAI.

[58]  Bernhard Schölkopf,et al.  Inferring deterministic causal relations , 2010, UAI.

[59]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[60]  Bernhard Schölkopf,et al.  Nonlinear causal discovery with additive noise models , 2008, NIPS.

[61]  Kevin P. Murphy,et al.  Bayesian structure learning using dynamic programming and MCMC , 2007, UAI.

[62]  Bernhard Schölkopf,et al.  A kernel-based causal learning algorithm , 2007, ICML '07.

[63]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems - Exact Computational Methods for Bayesian Networks , 1999, Information Science and Statistics.

[64]  Aapo Hyvärinen,et al.  A Linear Non-Gaussian Acyclic Model for Causal Discovery , 2006, J. Mach. Learn. Res..

[65]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[66]  Frederick Eberhardt,et al.  N-1 Experiments Suffice to Determine the Causal Relations Among N Variables , 2006 .

[67]  K. Sachs,et al.  Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data , 2005, Science.

[68]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[69]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[70]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[71]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[72]  Gregory F. Cooper,et al.  Causal Discovery from a Mixture of Experimental and Observational Data , 1999, UAI.

[73]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[74]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .