On the Failure of the Bootstrap for Matching Estimators

Matching estimators are widely used in empirical economics for the evaluation of programs or treatments. Researchers using matching methods often apply the bootstrap to calculate the standard errors. However, no formal justification has been provided for the use of the bootstrap in this setting. In this article, we show that the standard bootstrap is, in general, not valid for matching estimators, even in the simple case with a single continuous covariate where the estimator is root-N consistent and asymptotically normally distributed with zero asymptotic bias. Valid inferential methods in this setting are the analytic asymptotic variance estimator of Abadie and Imbens (2006a) as well as certain modifications of the standard bootstrap, like the subsampling methods in Politis and Romano (1994).

[1]  Carlos Matus,et al.  Escuela de gobierno , 2007 .

[2]  J. Davis.  Univariate Discrete Distributions , 2006 .

[3]  W. Newey,et al.  GMM with Many Weak Moment Conditions , 2005 .

[4]  Kosuke Imai,et al.  Do Get-Out-the-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field Experiments , 2005, American Political Science Review.

[5]  G. Imbens,et al.  Implementing Matching Estimators for Average Treatment Effects in Stata , 2004 .

[6]  Roberto Agodini,et al.  Are Experiments the Only Option? A Look at Dropout Prevention Programs , 2004, Review of Economics and Statistics.

[7]  M. Lauer,et al.  Impact of Donor Spontaneous Intracranial Hemorrhage on Outcome after Heart Transplantation , 2004, American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons.

[8]  Robert G. Olsen,et al.  The Impacts of Regular Upward Bound: Results from the Third Follow-Up Data Collection. Washington, DC: Mathematica Policy Research , 2004 .

[9]  G. Imbens,et al.  Large Sample Properties of Matching Estimators for Average Treatment Effects , 2004 .

[10]  B. Sianesi.  An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s , 2004, Review of Economics and Statistics.

[11]  M. Ravallion,et al.  Social Protection in a Crisis - Argentina's Plan Jefes Y Jefas , 2003 .

[12]  L. Guarcello,et al.  Household Vulnerability and Child Labor: The Effect of Shocks, Credit Rationing and Insurance , 2003 .

[13]  Sascha O. Becker,et al.  Estimation of Average Treatment Effects Based on Propensity Scores , 2002 .

[14]  Patrick a. Puhani.  Advantage Through Training in Poland? A Microeconometric Evaluation of the Employment Effects of Training and Job Subsidy Programmes , 2002 .

[15]  G. Imbens,et al.  Bias-Corrected Matching Estimators for Average Treatment Effects , 2011 .

[16]  Laura B. Rawlings,et al.  The Impact and Targeting of Social Infrastructure Investments , 2002 .

[17]  Michael Lechner,et al.  Some practical issues in the evaluation of heterogeneous labour market programmes by matching methods , 2002 .

[18]  Rajeev Dehejia,et al.  Propensity Score-Matching Methods for Nonexperimental Causal Studies , 2002, Review of Economics and Statistics.

[19]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[20]  Peter Hall,et al.  Bootstrap methods in statistics , 2000 .

[21]  G. Imbens,et al.  Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score , 2000 .

[22]  M. Ravallion,et al.  Income Gains to the Poor from Workfare: Estimates for Argentina's Trabajar Program , 1999 .

[23]  Patrick a. Puhani.  Advantage Through Training? A Microeconometric Evaluation of the Employment Effects of Active Labour Market Programmes in Poland , 1998 .

[24]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[25]  T. Shakespeare,et al.  Observational Studies , 2003 .

[26]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[27]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[28]  Paul G. Spirakis,et al.  Tail bounds for occupancy and the satisfiability threshold conjecture , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[29]  Joseph P. Romano,et al.  Large Sample Confidence Regions Based on Subsamples under Minimal Assumptions , 1994 .

[30]  A. W. Kemp,et al.  Univariate Discrete Distributions , 1993 .

[31]  E. Mammen,et al.  Comparing Nonparametric Versus Parametric Regression Fits , 1993 .

[32]  P. Hall.  The Bootstrap and Edgeworth Expansion , 1992 .

[33]  Paul R. Rosenbaum,et al.  Optimal Matching for Observational Studies , 1989 .

[34]  K. Athreya.  BOOTSTRAP OF THE MEAN IN THE INFINITE VARIANCE CASE , 1987 .

[35]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[36]  K. Joag-dev,et al.  Negative Association of Random Variables with Applications , 1983 .

[37]  D. Freedman,et al.  Some Asymptotic Theory for the Bootstrap , 1981 .

[38]  Norman L. Johnson,et al.  Urn models and their application , 1977 .

[39]  B. McCarl,et al.  Economics , 1870, The Indian medical gazette.