Randomization Tests in Observational Studies With Staggered Adoption of Treatment

ABSTRACT This article considers the problem of inference in observational studies with time-varying adoption of treatment. In addition to an unconfoundedness assumption that the potential outcomes are independent of the times at which units adopt treatment conditional on the units’ observed characteristics, our analysis assumes that the time at which each unit adopts treatment follows a Cox proportional hazards model. This assumption permits the time at which each unit adopts treatment to depend on the observed characteristics of the unit, but imposes the restriction that the probability of multiple units adopting treatment at the same time is zero. In this context, we study randomization tests of a null hypothesis that specifies that there is no treatment effect for all units and all time periods in a distributional sense. We first show that an infeasible test that treats the parameters of the Cox model as known has rejection probability under the null hypothesis no greater than the nominal level in finite samples. Since these parameters are unknown in practice, this result motivates a feasible test that replaces these parameters with consistent estimators. While the resulting test does not need to have the same finite-sample validity as the infeasible test, we show that it has limiting rejection probability under the null hypothesis no greater than the nominal level. In a simulation study, we examine the practical relevance of our theoretical results, including robustness to misspecification of the model for the time at which each unit adopts treatment. Finally, we provide an empirical application of our methodology using the synthetic control-based test statistic and tobacco legislation data found in Abadie, Diamond and Hainmueller. Supplementary materials for this article are available online.

[1]  Bruno Ferman,et al.  Synthetic controls with imperfect pretreatment fit , 2016, Quantitative Economics.

[2]  Joseph P. Romano,et al.  On the uniform asymptotic validity of subsampling and the bootstrap , 2012, 1204.2762.

[3]  P. Rosenbaum.  Sensitivity analysis for certain permutation inferences in matched observational studies , 1987 .

[4]  G. Imbens,et al.  Balancing, Regression, Difference-in-Differences and Synthetic Control Methods: A Synthesis , 2016, 1610.07748.

[5]  G. Imbens,et al.  Matrix Completion Methods for Causal Panel Data Models , 2017, Journal of the American Statistical Association.

[6]  Kathleen T. Li,et al.  Statistical Inference for Average Treatment Effects Estimated by Synthetic Control Methods , 2019, Journal of the American Statistical Association.

[7]  J. Cavanaugh,et al.  Partial Likelihood , 2018, Wiley StatsRef: Statistics Reference Online.

[8]  David Card,et al.  Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania , 1993 .

[9]  Yishu Xue,et al.  Diagnostics for the Cox model , 2017 .

[10]  Alan Clarke,et al.  A Panel Data Approach , 1999 .

[11]  J. Snow.  On the Mode of Communication of Cholera , 1856, Edinburgh medical journal.

[12]  Alberto Abadie,et al.  Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects , 2021, Journal of Economic Literature.

[13]  R. Gill,et al.  Cox's regression model for counting processes: a large sample study : (preprint) , 1982 .

[14]  Victor Chernozhukov,et al.  An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls , 2017, Journal of the American Statistical Association.

[15]  Sergio Firpo,et al.  Synthetic Control Method: Inference, Sensitivity Analysis and Confidence Sets , 2018, Journal of Causal Inference.

[16]  David A. Hirshberg,et al.  Synthetic Difference in Differences , 2018, 1812.09970.

[17]  Avi Feller,et al.  Synthetic Controls and Weighted Event Studies with Staggered Adoption , 2019, 1912.03290.

[18]  H. Mueller,et al.  The Economic Costs of Conflict , 2015 .

[19]  J. Barkoulas,et al.  Econometric Methods I , 2015 .

[20]  Muhammad J. Amjad,et al.  Robust Synthetic Control , 2017, J. Mach. Learn. Res..

[21]  Cheng Hsiao,et al.  A Panel Data Approach for Program Evaluation — Measuring the Benefits of Political and Economic Integration of Hong Kong with Mainland China , 2012 .

[22]  Richard Kay,et al.  Cox's Regression Model , 2004 .

[23]  Matias D. Cattaneo,et al.  Prediction Intervals for Synthetic Control Methods , 2021, Journal of the American Statistical Association.

[24]  Victor Chernozhukov,et al.  Practical and robust $t$-test based inference for synthetic control and related methods , 2018 .

[25]  P. Lewis.  Minimum Wages and Employment , 2006 .

[26]  Alberto Abadie,et al.  The Economic Costs of Conflict: A Case Study of the , 2003 .

[27]  Alberto Abadie,et al.  A Penalized Synthetic Control Estimator for Disaggregated Data , 2021, Journal of the American Statistical Association.

[28]  Laura Spierdijk,et al.  Nonparametric conditional hazard rate estimation: A local linear approach , 2008, Comput. Stat. Data Anal..

[29]  T. Shakespeare,et al.  Observational Studies , 2003 .

[30]  Susan Athey,et al.  Design-Based Analysis in Difference-in-Differences Settings with Staggered Adoption , 2018, Journal of Econometrics.

[31]  Matias D. Cattaneo,et al.  Econometric Methods for Program Evaluation , 2018, Annual Review of Economics.

[32]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[33]  W. Orzechowski,et al.  Tax Burden on Tobacco , 2001 .

[34]  J. Kalbfleisch,et al.  The Statistical Analysis of Failure Time Data , 1980 .