Synthetic controls with imperfect pretreatment fit

We analyze the properties of the Synthetic Control (SC) and related estimators when the pre‐treatment fit is imperfect. In this framework, we show that these estimators are generally biased if treatment assignment is correlated with unobserved confounders, even when the number of pre‐treatment periods goes to infinity. Still, we show that a demeaned version of the SC method can improve in terms of bias and variance relative to the difference‐in‐difference estimator. We also derive a specification test for the demeaned SC estimator in this setting with imperfect pre‐treatment fit. Given our theoretical results, we provide practical guidance for applied researchers on how to justify the use of such estimators in empirical applications.

[1]  Bruno Ferman,et al.  On the role of covariates in the synthetic control method , 2019, The Econometrics Journal.

[2]  Laurent Gobillon,et al.  Regional Policy Evaluation: Interactive Fixed Effects and Synthetic Controls , 2014, Review of Economics and Statistics.

[3]  M. Weidner,et al.  Linear Regression for Panel with Unknown Number of Factors as Interactive Fixed Effects , 2014 .

[4]  Victor Chernozhukov,et al.  Practical and robust $t$-test based inference for synthetic control and related methods , 2018 .

[5]  Peter C. B. Phillips,et al.  Optimal Inference in Cointegrated Systems , 1991 .

[6]  David A. Hirshberg,et al.  Synthetic Difference in Differences , 2018, 1812.09970.

[7]  P. Phillips,et al.  Asymptotic Properties of Residual Based Tests for Cointegration , 1990 .

[8]  J. Bai,et al.  Determining the Number of Factors in Approximate Factor Models , 2000 .

[9]  Avi Feller,et al.  The Augmented Synthetic Control Method , 2018, Journal of the American Statistical Association.

[10]  Yiqing Xu.  Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models , 2016, Political Analysis.

[11]  Ricardo P. Masini,et al.  The Perils of Counterfactual Analysis with Integrated Processes , 2016 .

[12]  Victor Chernozhukov,et al.  An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls , 2017, Journal of the American Statistical Association.

[13]  Alberto Abadie,et al.  Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects , 2021, Journal of Economic Literature.

[14]  E. Pesavento.  Analytical Evaluation of the Power of Tests for the Absence of Cointegration , 2000 .

[15]  Gregor Pfeifer,et al.  Synthetic Control Methods: Never Use All Pre-Intervention Outcomes as Economic Predictors , 2016 .

[16]  Pedro H. C. Sant'Anna,et al.  Difference-in-Differences with Multiple Time Periods , 2018, Journal of Econometrics.

[17]  Kathleen T. Li,et al.  Estimation of average treatment effects with panel data: Asymptotic theory and implementation , 2017 .

[18]  Xavier d'Haultfoeuille,et al.  Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects , 2018, American Economic Review.

[19]  V. Chernozhukov,et al.  Estimation and Confidence Regions for Parameter Sets in Econometric Models , 2007 .

[20]  Susan Athey,et al.  The State of Applied Econometrics - Causality and Policy Evaluation , 2016, 1607.00699.

[21]  Andrew Goodman-Bacon.  Difference-in-Differences with Variation in Treatment Timing , 2018, Journal of Econometrics.

[22]  W. Fuller,et al.  LIKELIHOOD RATIO STATISTICS FOR AUTOREGRESSIVE TIME SERIES WITH A UNIT ROOT , 1981 .

[23]  Alberto Abadie,et al.  The Economic Costs of Conflict: A Case Study of the , 2003 .

[24]  Jens Hainmueller,et al.  Comparative Politics and the Synthetic Control Method , 2014 .

[25]  Ricardo P. Masini,et al.  Counterfactual Analysis With Artificial Controls: Inference, High Dimensions and Nonstationarity , 2019 .

[26]  J. Bai,et al.  Inferential Theory for Factor Models of Large Dimensions , 2003 .

[27]  G. Imbens,et al.  Balancing, Regression, Difference-in-Differences and Synthetic Control Methods: A Synthesis , 2016, 1610.07748.

[28]  Muhammad J. Amjad,et al.  Robust Synthetic Control , 2017, J. Mach. Learn. Res..

[29]  Cheng Hsiao,et al.  A Panel Data Approach for Program Evaluation — Measuring the Benefits of Political and Economic Integration of Hong Kong with Mainland China , 2012 .

[30]  Bruno Ferman,et al.  On the Properties of the Synthetic Control Estimator with Many Periods and Many Controls , 2019, 1906.06665.

[31]  Susan Athey,et al.  Design-based analysis in Difference-In-Differences settings with staggered adoption , 2021 .

[32]  J. Bai,et al.  Panel Data Models With Interactive Fixed Effects , 2009 .

[33]  Bruno Ferman,et al.  Cherry Picking with Synthetic Controls , 2020 .

[34]  W. Newey,et al.  Uniform Convergence in Probability and Stochastic Equicontinuity , 1991 .

[35]  G. Imbens,et al.  Matrix Completion Methods for Causal Panel Data Models , 2017, Journal of the American Statistical Association.

[36]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[37]  S. Johansen.  Estimation and Hypothesis Testing of Cointegration Vectors in Gaussian Vector Autoregressive Models , 1991 .

[38]  Rob J Hyndman,et al.  Automatic Time Series Forecasting: The forecast Package for R , 2008 .

[39]  W. Newey,et al.  Large sample estimation and hypothesis testing , 1986 .

[40]  James D. Hamilton.  Time Series Analysis , 1994 .

[41]  David Powell,et al.  Imperfect Synthetic Controls: Did the Massachusetts Health Care Reform Save Lives? , 2018 .

[42]  Susan Athey,et al.  Design-Based Analysis in Difference-in-Differences Settings with Staggered Adoption , 2018, Journal of Econometrics.

[43]  P. Perron,et al.  Lag Length Selection and the Construction of Unit Root Tests with Good Size and Power , 2001 .