Difference-in-Differences with Multiple Time Periods

Difference-in-Differences (DID) is one of the most important and popular designs for evaluating causal effects of policy changes. In its standard format, there are two time periods and two groups: in the first period no one is treated, and in the second period a "treatment group" becomes treated, whereas a "control group" remains untreated. However, many em- pirical applications of the DID design have more than two periods and variation in treatment timing. In this article, we consider identification and estimation of treatment effect parameters using DID with (i) multiple time periods, (ii) variation in treatment timing, and (iii) when the "parallel trends assumption" holds potentially only after conditioning on observed covariates. We propose a simple two-step estimation strategy, establish the asymptotic prop- erties of the proposed estimators, and prove the validity of a computationally convenient bootstrap procedure. Furthermore we propose a semiparametric data-driven testing procedure to assess the credibility of the DID design in our context. Finally, we analyze the effect of the minimum wage on teen employment from 2001-2007. By using our proposed methods we confront the challenges related to variation in the timing of the state-level minimum wage policy changes. Open-source software is available for implementing the proposed methods.

[1]  Minimum Wage Effects Across State Borders: Estimates Using Contiguous Counties , 2007 .

[2]  Wenceslao González-Manteiga,et al.  An updated review of Goodness-of-Fit tests for regression models , 2013, TEST.

[3]  Population in Perspective. , 1969 .

[4]  Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects , 2019 .

[5]  Tong Li,et al.  Quantile Treatment Effects in Difference in Differences Models with Panel Data , 2019, Quantitative Economics.

[6]  Xiaoxia Shi,et al.  Inference Based on Conditional Moment Inequalities , 2010 .

[7]  Susan Athey,et al.  Design-Based Analysis in Difference-in-Differences Settings with Staggered Adoption , 2018, Journal of Econometrics.

[8]  Sukjin Han Identification in Nonparametric Models for Dynamic Treatment Effects , 2018, Journal of Econometrics.

[9]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[10]  Tymon S loczy 'nski A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands , 2018 .

[11]  F. Windmeijer,et al.  Estimation of Panel Data Models with Binary Indicators When Treatment Effects are Not Constant Over Time , 2004 .

[12]  J M Robins,et al.  Marginal Mean Models for Dynamic Regimes , 2001, Journal of the American Statistical Association.

[13]  J. Wooldridge Fixed-Effects and Related Estimators for Correlated Random-Coefficient and Treatment-Effect Panel Data Models , 2005, Review of Economics and Statistics.

[14]  Matías Busso,et al.  New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators , 2014, Review of Economics and Statistics.

[15]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[16]  J. Wooldridge Inverse probability weighted estimation for general missing data problems , 2004 .

[17]  Oliver Linton,et al.  Estimation of Semiparametric Models When the Criterion Function is Not Smooth , 2002 .

[18]  Kosuke Imai,et al.  Matching Methods for Causal Inference with Time‐Series Cross‐Sectional Data , 2021, American Journal of Political Science.

[19]  Pedro H. C. Sant'Anna,et al.  Specification Tests for the Propensity Score , 2019 .

[20]  Michael Reich,et al.  Minimum Wage Shocks, Employment Flows, and Labor Market Frictions , 2012, Journal of Labor Economics.

[21]  R. Blundell,et al.  Evaluating the Employment Impact of a Mandatory Job Search Programme , 2003 .

[22]  D. Neumark,et al.  Evidence on Employment Effects of Minimum Wages and Subminimum Wage Provisions from Panel Data on State Minimum Wage Laws , 1991 .

[23]  Xiaohong Chen Chapter 76 Large Sample Sieve Estimation of Semi-Nonparametric Models , 2007 .

[24]  H. Bierens Consistent model specification tests , 1982 .

[25]  J. K. Ghosh,et al.  On the Logical Foundations of Survey Sampling , 1988 .

[26]  D. Horvitz,et al.  A Generalization of Sampling Without Replacement from a Finite Universe , 1952 .

[27]  W. Newey,et al.  The asymptotic variance of semiparametric estimators , 1994 .

[28]  J. Wooldridge VIOLATING IGNORABILITY OF TREATMENT BY CONTROLLING FOR TOO MANY FACTORS , 2005, Econometric Theory.

[29]  W. Newey,et al.  Large sample estimation and hypothesis testing , 1986 .

[30]  J. Schmitt Why Does the Minimum Wage Have No Discernible Effect on Employment , 2013 .

[31]  M. Reich,et al.  Minimum Wage Effects Across State Borders: Estimates Using Contiguous Counties , 2007, The Review of Economics and Statistics.

[32]  D. Rubin The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials , 2007, Statistics in medicine.

[33]  Joshua D. Angrist,et al.  Mostly Harmless Econometrics: An Empiricist's Companion , 2008 .

[34]  Neil Shephard,et al.  Panel experiments and dynamic causal effects: A finite population perspective , 2020, Quantitative Economics.

[35]  B. Graham,et al.  Inverse Probability Tilting for Moment Condition Models with Missing Data , 2008 .

[36]  David Autor,et al.  Do Employment Protections Reduce Productivity? Evidence from U.S. States , 2007, SSRN Electronic Journal.

[37]  E. Mammen Bootstrap and Wild Bootstrap for High Dimensional Linear Models , 1993 .

[38]  Stéphane Bonhomme,et al.  Recovering Distributions in Difference-in-Differences Models: A Comparison of Selective and Comprehensive Schooling , 2011, Review of Economics and Statistics.

[39]  Jing Qin,et al.  Empirical‐likelihood‐based difference‐in‐differences estimators , 2008 .

[40]  Patrick M. Kline,et al.  A Score Based Approach to Wild Bootstrap Inference , 2010 .

[41]  Bruno Ferman,et al.  Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity , 2019, Review of Economics and Statistics.

[42]  T. Stanley,et al.  Publication Selection Bias in Minimum-Wage Research? A Meta-Regression Analysis , 2009 .

[43]  Gerard J. van den Berg,et al.  The nonparametric identification of treatment effects in duration models , 2003 .

[44]  D. Rubin For objective causal inference, design trumps analysis , 2008, 0811.1640.

[45]  Matthew D. Webb,et al.  The Wild Bootstrap for Few (Treated) Clusters , 2018 .

[46]  Xavier d'Haultfoeuille,et al.  Fuzzy Differences-in-Differences , 2015, 1510.01757.

[47]  Tymon Sloczy'nski,et al.  A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands , 2018, 1810.01576.

[48]  D. Neumark,et al.  Revisiting the Minimum Wage—Employment Debate: Throwing Out the Baby with the Bathwater? , 2013 .

[49]  David Autor,et al.  Does Employment Protection Reduce Productivity? Evidence from U.S. States , 2007 .

[50]  Xavier D’Haultfœuille,et al.  Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects , 2019 .

[51]  Shakeeb Khan,et al.  Irregular Identification, Support Conditions, and Inverse Weight Estimation , 2010 .

[52]  Jeffrey M. Wooldridge,et al.  Cluster-Sample Methods in Applied Econometrics , 2003 .

[53]  H. Farber Employment, Hours, and Earnings Consequences of Job Loss: US Evidence from the Displaced Workers Survey , 2017, Journal of Labor Economics.

[54]  Andrew Goodman-Bacon Difference-in-Differences with Variation in Treatment Timing , 2018, Journal of Econometrics.

[55]  Richard K. Crump,et al.  Dealing with limited overlap in estimation of average treatment effects , 2009 .

[56]  Xiaohong Chen,et al.  Estimation of Semiparametric Models When the Criterion Function is Not Smooth , 2002 .

[57]  B. Sianesi An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s , 2004, Review of Economics and Statistics.

[58]  M. Kosorok Introduction to Empirical Processes and Semiparametric Inference , 2008 .

[59]  W. Newey,et al.  Average and Quantile Effects in Nonseparable Panel Models , 2009, 0904.1990.

[60]  Pedro H. C. Sant’Anna Nonparametric Tests for Treatment Effect Heterogeneity With Duration Outcomes , 2016, Journal of Business & Economic Statistics.

[61]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[62]  Joachim Freyberger,et al.  Uniform confidence bands: Characterization and optimality , 2018 .

[63]  J. Kmenta Mostly Harmless Econometrics: An Empiricist's Companion , 2010 .

[64]  J. Robins A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect , 1986 .

[65]  E. Duflo,et al.  How Much Should We Trust Differences-in-Differences Estimates? , 2001 .

[66]  Pedro H. C. Sant'Anna,et al.  Doubly robust difference-in-differences estimators , 2020 .

[67]  S. Chabé-Ferret Analysis of the bias of Matching and Difference-in-Difference under alternative earnings and selection processes , 2015 .

[68]  Federico H. Gutierrez,et al.  Difference-in-Differences When Treatment Status Is Observed in Only One Period , 2014 .

[69]  Stephen G. Donald,et al.  Estimation and inference for distribution functions and quantile functions in treatment effect models , 2014 .

[70]  S. Murphy,et al.  Optimal dynamic treatment regimes , 2003 .

[71]  Pedro H. C. Sant'Anna,et al.  Program Evaluation with Right-Censored Data , 2016, 1604.02642.

[72]  E. Giné,et al.  Bootstrapping General Empirical Measures , 1990 .

[73]  J. Escanciano Joint and marginal specification tests for conditional mean and variance models , 2008 .

[74]  Victor Chernozhukov,et al.  The sorted effects method: discovering heterogeneous effects beyond their averages , 2015 .

[75]  Liyang Sun,et al.  Estimating Dynamic Treatment Effects in Event Studies With Heterogeneous Treatment Effects , 2018, Journal of Econometrics.

[76]  Laura A. Hatfield,et al.  Matching and Regression to the Mean in Difference‐in‐Differences Analysis , 2017, Health services research.

[77]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[78]  Michael B. Urbancic,et al.  Broken or Fixed Effects , 2019 .

[79]  P. Ding,et al.  Asymptotic inference of causal effects with observational studies trimmed by the estimated propensity scores , 2018 .

[80]  J. Heckman,et al.  Dynamic Treatment Effects. , 2016, Journal of econometrics.

[81]  M. Sherman,et al.  A comparison between bootstrap methods and generalized estimating equations for correlated outcomes in generalized linear models , 1997 .

[82]  Maxwell B. Stinchcombe,et al.  CONSISTENT SPECIFICATION TESTING WITH NUISANCE PARAMETERS PRESENT ONLY UNDER THE ALTERNATIVE , 1998, Econometric Theory.

[83]  J. Escanciano A CONSISTENT DIAGNOSTIC TEST FOR REGRESSION MODELS USING PROJECTIONS , 2006, Econometric Theory.

[84]  Xavier Jaravel,et al.  Revisiting Event Study Designs , 2017 .

[85]  Jose Luis Montiel Olea,et al.  Appendix For : “ Simultaneous Confidence Bands : Theory , Implementation , and an Application to SVARs ” , 2018 .

[86]  S. Nath FROM A FINITE UNIVERSE , 2016 .

[87]  Jacob L. Vigdor,et al.  Minimum Wage Increases, Wages, and Low-Wage Employment: Evidence from Seattle , 2017 .

[88]  J. Roth Pre-test with Caution: Event-study Estimates After Testing for Parallel Trends , 2019 .

[89]  Winfried Stute,et al.  Nonparametric model checks for regression , 1997 .

[90]  M. Bailey,et al.  The War on Poverty&Apos;S Experiment in Public Medicine: Community Health Centers and the Mortality of Older Americans , 2014, The American economic review.

[91]  Herman J. Bierens,et al.  Asymptotic Theory of Integrated Conditional Moment Tests , 1997 .

[92]  J. Robins,et al.  Comment: Performance of Double-Robust Estimators When “Inverse Probability” Weights Are Highly Variable , 2007, 0804.2965.

[93]  A. Belloni,et al.  Program evaluation and causal inference with high-dimensional data , 2013, 1311.2645.

[94]  A. Heisz,et al.  The Short- and Long-Term Career Effects of Graduating in a Recession , 2012 .

[95]  Xiaohong Chen,et al.  Semiparametric efficiency in GMM models with auxiliary data , 2007, 0705.0069.

[96]  Pedro H. C. Sant'Anna,et al.  The Role of Parallel Trends in Event Study Settings: An Application to Environmental Economics , 2020, Journal of the Association of Environmental and Resource Economists.

[98]  M. Lechner The Estimation of Causal Effects by Difference-in-Difference Methods , 2011 .

[99]  Matthew D. Webb,et al.  Randomization inference for difference-in-differences with few treated clusters , 2020 .

[100]  David Card,et al.  Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania , 1993 .

[101]  S. Chabé-Ferret Should We Combine Difference In Differences with Conditioning on Pre-Treatment Outcomes? , 2017 .

[102]  J. Robins Addendum to “a new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect” , 1987 .

[103]  Guang Cheng,et al.  The cluster bootstrap consistency in generalized estimating equations , 2013, J. Multivar. Anal..

[104]  Timothy G. Conley,et al.  Inference with “Difference in Differences” with a Small Number of Policy Changes , 2005, The Review of Economics and Statistics.

[105]  J. Carlos Escanciano,et al.  Goodness-of-Fit Tests for Linear and Nonlinear Time Series Models , 2006 .

[106]  Pedro H. C. Sant'Anna,et al.  Specification Tests for the Propensity Score , 2016, Journal of Econometrics.

[107]  Susan Athey,et al.  When Should You Adjust Standard Errors for Clustering? , 2017, The Quarterly Journal of Economics.

[108]  J. Reif,et al.  Interpreting Pre-Trends as Anticipation: Impact on Estimated Treatment Effects from Tort Reform , 2015 .

[109]  Tong Li,et al.  Quantile Treatment Effects in Difference in Differences Models Under Dependence Restrictions and with Only Two Time Periods , 2017, Journal of Econometrics.

[110]  Susan Athey,et al.  Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis , 2017, Econometrica.