Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models

While probabilistic models are an important tool for studying causality, doing so suffers from the intractability of inference. As a step towards tractable causal models, we consider the problem of learning interventional distributions using sum-product networks (SPNs) that are over-parameterized by gate functions, e.g., neural networks. Providing an arbitrarily intervened causal graph as input, effectively subsuming Pearl’s do-operator, the gate function predicts the parameters of the SPN. The resulting interventional SPNs are motivated and illustrated by a structural causal model themed around personal health. Our empirical evaluation on three benchmark data sets as well as a synthetic health data set clearly demonstrates that interventional SPNs indeed are both expressive in modelling and flexible in adapting to the interventions.

[1]  E. H. Simpson,et al.  The Interpretation of Interaction in Contingency Tables , 1951 .

[2]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[3]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[4]  Dan Roth,et al.  On the Hardness of Approximate Reasoning , 1993, IJCAI.

[5]  S. Srihari.  Mixture Density Networks , 1994 .

[6]  Didier Dubois,et al.  Mathematical models for handling partial knowledge in artificial intelligence , 1995 .

[7]  Judea Pearl,et al.  From Bayesian networks to causal networks , 1995 .

[8]  C. Granger.  Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[9]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[10]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[11]  Daniel Zelterman,et al.  Bayesian Artificial Intelligence , 2005, Technometrics.

[12]  A. Gopnik,et al.  Causal learning : psychology, philosophy, and computation , 2007 .

[13]  Steven A. Sloman,et al.  Causal reasoning through intervention , 2007 .

[14]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[15]  Pedro M. Domingos,et al.  Sum-product networks: A new deep architecture , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[16]  Guy Van den Broeck,et al.  Probabilistic Sentential Decision Diagrams , 2014, KR.

[17]  Hugo Larochelle,et al.  MADE: Masked Autoencoder for Distribution Estimation , 2015, ICML.

[18]  Han Zhao,et al.  On the Relationship between Sum-Product Networks and Bayesian Networks , 2015, ICML.

[19]  Alexandros G. Dimakis,et al.  Learning Causal Graphs with Small Interventions , 2015, NIPS.

[20]  Mihaela van der Schaar,et al.  Bounded Off-Policy Evaluation with Missing Data for Course Recommendation and Curriculum Design , 2016, ICML.

[21]  J. Pearl,et al.  Causal Inference in Statistics: A Primer , 2016 .

[22]  Elias Bareinboim,et al.  Causal inference and the data-fusion problem , 2016, Proceedings of the National Academy of Sciences.

[23]  R. Eisinger,et al.  A causal Bayesian network model of disease progression mechanisms in chronic myeloid leukemia. , 2017, Journal of theoretical biology.

[24]  Nick Chater,et al.  Causal Models and Conditional Reasoning , 2017 .

[25]  Michael R. Waldmann,et al.  The Oxford handbook of causal reasoning , 2017 .

[26]  Pascal Poupart,et al.  Online Structure Learning for Feed-Forward and Recurrent Sum-Product Networks , 2018, NeurIPS.

[27]  Amnon Shashua,et al.  Sum-Product-Quotient Networks , 2018, AISTATS.

[28]  Adnan Darwiche,et al.  On the Relative Expressiveness of Bayesian and Neural Networks , 2018, PGM.

[29]  Constantinos Daskalakis,et al.  Learning and Testing Causal Models with Interventions , 2018, NeurIPS.

[30]  Bernhard Schölkopf,et al.  Learning causality and causality-related learning: some recent progress. , 2018, National science review.

[31]  Daniel C. Castro,et al.  Morpho-MNIST: Quantitative Assessment and Diagnostics for Representation Learning , 2018, J. Mach. Learn. Res..

[32]  Nan Rosemary Ke,et al.  Learning Neural Causal Models from Unknown Interventions , 2019, ArXiv.

[33]  Judea Pearl,et al.  The seven tools of causal inference, with reflections on machine learning , 2019, Commun. ACM.

[34]  Bernhard Scholkopf.  Causality for Machine Learning , 2019 .

[35]  David Sontag,et al.  Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Models , 2019, ICML.

[36]  Zeb Kurth-Nelson,et al.  Causal Reasoning from Meta-reinforcement Learning , 2019, ArXiv.

[37]  Joseph Y. Halpern,et al.  Abstracting Causal Models , 2018, AAAI.

[38]  Franz Pernkopf,et al.  Bayesian Learning of Sum-Product Networks , 2019, NeurIPS.

[39]  Kristian Kersting,et al.  Random Sum-Product Networks: A Simple and Effective Approach to Probabilistic Deep Learning , 2019, UAI.

[40]  Amit Sharma,et al.  DoWhy: An End-to-End Library for Causal Inference , 2020, ArXiv.

[41]  Vaishak Belle,et al.  Learning Credal Sum-Product Networks , 2020, AKBC.

[42]  Alexandre Lacoste,et al.  Differentiable Causal Discovery from Interventional Data , 2020, NeurIPS.

[43]  Kristian Kersting,et al.  Conditional Sum-Product Networks: Imposing Structure on Deep Probabilistic Architectures , 2019, PGM.

[44]  Jeong-Yoon Lee,et al.  CausalML: Python Package for Causal Machine Learning , 2020, ArXiv.

[45]  J. F. Carriger,et al.  A Bayesian network approach to refining ecological risk assessments: Mercury and the Florida panther (Puma concolor coryi). , 2020, Ecological modelling.

[46]  N. Feroze.  Forecasting the patterns of COVID-19 and causal impacts of lockdown in top five affected countries using Bayesian Structural Time Series Models , 2020, Chaos, Solitons & Fractals.

[47]  Guy Van den Broeck,et al.  Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits , 2020, ICML.

[48]  Mihaela van der Schaar,et al.  Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders , 2019, ICML.

[49]  N. Fenton,et al.  COVID-19 infection and death rates: the need to incorporate causal explanations for the data and avoid bias in testing , 2020, Journal of Risk Research.

[50]  Vaishak Belle,et al.  Interventions and Counterfactuals in Tractable Probabilistic Models: Limitations of Contemporary Transformations , 2020, ArXiv.

[51]  Yoshua Bengio,et al.  The Causal-Neural Connection: Expressiveness, Learnability, and Inference , 2021, ArXiv.

[52]  Marko Sarstedt,et al.  Data, measurement, and causal inferences in machine learning: opportunities and challenges for marketing , 2021 .

[53]  Iago Par'is,et al.  Sum-Product Networks: A Survey , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.