Causal Inference using Gaussian Processes with Structured Latent Confounders

Latent confounders---unobserved variables that influence both treatment and outcome---can bias estimates of causal effects. In some cases, these confounders are shared across observations, e.g. all students taking a course are influenced by the course's difficulty in addition to any educational interventions they receive individually. This paper shows how to semiparametrically model latent confounders that have this structure and thereby improve estimates of causal effects. The key innovations are a hierarchical Bayesian model, Gaussian processes with structured latent confounders (GP-SLC), and a Monte Carlo inference algorithm for this model based on elliptical slice sampling. GP-SLC provides principled Bayesian uncertainty estimates of individual treatment effect with minimal assumptions about the functional forms relating confounders, covariates, treatment, and outcome. Finally, this paper shows GP-SLC is competitive with or more accurate than widely used causal inference techniques on three benchmark datasets, including the Infant Health and Development Program and a dataset showing the effect of changing temperatures on state-wide energy consumption across New England.

[1]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[2]  Jonathan A. Batten,et al.  Scaling Relationships of Gaussian Processes , 2001 .

[3]  M. Masson,et al.  Using confidence intervals in within-subject designs , 1994, Psychonomic bulletin & review.

[4]  Dustin Tran,et al.  Implicit Causal Models for Genome-wide Association Studies , 2017, ICLR.

[5]  Neil D. Lawrence,et al.  Bayesian Gaussian Process Latent Variable Model , 2010, AISTATS.

[6]  Ryan P. Adams,et al.  Elliptical slice sampling , 2009, AISTATS.

[7]  Max Welling,et al.  Causal Effect Inference with Deep Latent-Variable Models , 2017, NIPS 2017.

[8]  Judea Pearl,et al.  The algorithmization of counterfactuals , 2011, Annals of Mathematics and Artificial Intelligence.

[9]  Nando de Freitas,et al.  An Introduction to Sequential Monte Carlo Methods , 2001, Sequential Monte Carlo Methods in Practice.

[10]  Christopher Winship,et al.  Endogenous Selection Bias: The Problem of Conditioning on a Collider Variable. , 2014, Annual review of sociology.

[11]  Andrew Gelman,et al.  Multilevel (Hierarchical) Modeling: What It Can and Cannot Do , 2006, Technometrics.

[12]  J. Pearl,et al.  Measurement bias and effect restoration in causal inference , 2014 .

[13]  Z. Geng,et al.  Identifying Causal Effects With Proxy Variables of an Unmeasured Confounder. , 2016, Biometrika.

[14]  Suchi Saria,et al.  Reliable Decision Support using Counterfactual Models , 2017, NIPS.

[15]  Amanda Gentzel,et al.  The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data , 2019, NeurIPS.

[16]  Freda Kemp,et al.  An Introduction to Sequential Monte Carlo Methods , 2003 .

[17]  Ahmed M. Alaa,et al.  Bayesian Nonparametric Causal Inference: Information Rates and Learning Algorithms , 2017, IEEE Journal of Selected Topics in Signal Processing.

[18]  Guanglei Hong,et al.  Effects of kindergarten retention on children's social-emotional development: an application of propensity score method to multivariate, multilevel data. , 2008, Developmental psychology.

[19]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[20]  Bernhard Schölkopf,et al.  Invariant Gaussian Process Latent Variable Models and Application in Causal Discovery , 2010, UAI.

[21]  David M. Blei,et al.  The Blessings of Multiple Causes , 2018, Journal of the American Statistical Association.

[22]  Neil D. Lawrence,et al.  Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data , 2003, NIPS.

[23]  J BERKSON,et al.  Limitations of the application of fourfold table analysis to hospital data. , 1946, Biometrics.

[24]  Uri Shalit,et al.  Learning Representations for Counterfactual Inference , 2016, ICML.

[25]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[26]  D. Rubin,et al.  Causal Inference for Statistics, Social, and Biomedical Sciences: A General Method for Estimating Sampling Variances for Standard Estimators for Average Causal Effects , 2015 .

[27]  L M LaVange,et al.  Infant Health and Development Program for low birth weight, premature infants: program elements, family participation, and child intelligence. , 1992, Pediatrics.

[28]  David D. Jensen,et al.  Object Conditioning for Causal Inference , 2019, UAI.

[29]  Ricardo Silva,et al.  Gaussian Process Structural Equation Models with Latent Variables , 2010, UAI.

[30]  S. Zeger,et al.  Longitudinal data analysis using generalized linear models , 1986 .

[31]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[32]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[33]  S. Raudenbush,et al.  Evaluating Kindergarten Retention Policy , 2006 .

[34]  Mihaela van der Schaar,et al.  Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes , 2017, NIPS.

[35]  Alexander D'Amour,et al.  On Multi-Cause Approaches to Causal Inference with Unobserved Counfounding: Two Cautionary Failure Cases and A Promising Alternative , 2019, AISTATS.

[36]  Uri Shalit,et al.  Estimating individual treatment effect: generalization bounds and algorithms , 2016, ICML.

[37]  Vikash K. Mansinghka,et al.  Gen: a general-purpose probabilistic programming system with programmable inference , 2019, PLDI.

[38]  Jennifer L. Hill,et al.  Bayesian Nonparametric Modeling for Causal Inference , 2011 .

[39]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[40]  Peter M. Steiner,et al.  Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments , 2008 .

[41]  L. Peltonen,et al.  Classical twin studies and beyond , 2002, Nature Reviews Genetics.

[42]  David D. Jensen,et al.  Bayesian causal inference via probabilistic program synthesis , 2019, ArXiv.