Random Forests Approach for Causal Inference with Clustered Observational Data.

There is a growing interest in using machine learning (ML) methods for causal inference due to their (nearly) automatic and flexible ability to model key quantities such as the propensity score or the outcome model. Unfortunately, most ML methods for causal inference have been studied under single-level settings where all individuals are independent of each other and there is little work in using these methods with clustered or nested data, a common setting in education studies. This paper investigates using one particular ML method based on random forests known as Causal Forests to estimate treatment effects in multilevel observational data. We conduct simulation studies under different types of multilevel data, including two-level, three-level, and cross-classified data. Our simulation study shows that when the ML method is supplemented with estimated propensity scores from multilevel models that account for clustered/hierarchical structure, the modified ML method outperforms preexisting methods in a wide variety of settings. We conclude by estimating the effect of private math lessons in the Trends in International Mathematics and Science Study data, a large-scale educational assessment where students are nested within schools.

[1]  Bruno Arpino,et al.  The specification of the propensity score in multilevel observational studies , 2011, Comput. Stat. Data Anal..

[2]  Kosuke Imai,et al.  Causal Inference With General Treatment Regimes , 2004 .

[3]  G. Imbens.  The Role of the Propensity Score in Estimating Dose-Response Functions , 1999 .

[4]  Luke Keele,et al.  Estimation Methods for Cluster Randomized Trials with Noncompliance: A Study of A Biometric Smartcard Payment System in India , 2018, 1805.03744.

[5]  Jennifer L. Hill,et al.  Bayesian Nonparametric Modeling for Causal Inference , 2011 .

[6]  J. Schafer,et al.  Average causal effects from nonrandomized studies: a practical guide and simulated example. , 2008, Psychological methods.

[7]  S Natasha Beretvas,et al.  The Impact of Inappropriate Modeling of Cross-Classified Data Structures , 2006, Multivariate behavioral research.

[8]  J. Lunceford,et al.  Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study , 2004, Statistics in medicine.

[9]  Susan Athey,et al.  Estimation Considerations in Contextual Bandits , 2017, ArXiv.

[10]  D. Rubin.  Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation , 2001, Health Services and Outcomes Research Methodology.

[11]  Guanglei Hong,et al.  Marginal Mean Weighting Through Stratification: Adjustment for Selection Bias in Multilevel Data , 2010 .

[12]  Lihong Li,et al.  Provable Optimal Algorithms for Generalized Linear Contextual Bandits , 2017, ArXiv.

[13]  Neural Networks for Propensity Score Estimation: Simulation Results and Recommendations , 2015 .

[14]  Peter M. Steiner,et al.  The importance of covariate selection in controlling for selection bias in observational studies. , 2010, Psychological methods.

[15]  Stefan Wager,et al.  Estimation and Inference of Heterogeneous Treatment Effects using Random Forests , 2015, Journal of the American Statistical Association.

[16]  J. Robins,et al.  Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models , 1999 .

[17]  Daniel Westreich,et al.  Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regression. , 2010, Journal of clinical epidemiology.

[18]  Elizabeth A Stuart,et al.  Matching methods for causal inference: A review and a look forward. , 2010, Statistical science : a review journal of the Institute of Mathematical Statistics.

[19]  Susan Athey,et al.  Recursive partitioning for heterogeneous causal effects , 2015, Proceedings of the National Academy of Sciences.

[20]  D. Rubin.  Comment: Which Ifs Have Causal Answers , 1986 .

[21]  S. Athey,et al.  Generalized random forests , 2016, The Annals of Statistics.

[22]  Felix J Thoemmes,et al.  The Use of Propensity Scores for Nonrandomized Designs With Clustered Data , 2011, Multivariate behavioral research.

[23]  S. Raudenbush,et al.  Heterogeneous Agents, Social Interactions, and Causal Inference , 2013 .

[24]  Peter M. Steiner,et al.  Can Nonrandomized Experiments Yield Accurate Answers? A Randomized Experiment Comparing Random and Nonrandom Assignments , 2008 .

[25]  D. McCaffrey,et al.  Propensity score estimation with boosted regression for evaluating causal effects in observational studies. , 2004, Psychological methods.

[26]  Guanglei Hong,et al.  Reading Instruction Time and Homogeneous Grouping in Kindergarten: An Application of Marginal Mean Weighting Through Stratification , 2009 .

[27]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[28]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[29]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[30]  M. Seltzer,et al.  A Two-Stage Propensity Score Matching Strategy for Treatment Effect Estimation in a Multisite Observational Study , 2014 .

[31]  S. Raudenbush,et al.  Evaluating Kindergarten Retention Policy , 2006 .

[32]  Jee-Seon Kim,et al.  Multilevel Propensity Score Methods for Estimating Causal Effects: A Latent Class Modeling Strategy , 2015 .

[33]  Massimo Cannas,et al.  Propensity score matching with clustered data. An application to the estimation of the impact of caesarean section on the Apgar score , 2016, Statistics in medicine.

[34]  J. Robins,et al.  Estimation of Regression Coefficients When Some Regressors are not Always Observed , 1994 .