Estimation of Treatment Effects Without an Exclusion Restriction with an Application to the Analysis of the School Breakfast Program

While the rise in childhood obesity is clear, the policy ramifications are not. School nutrition programs such as the School Breakfast Program (SBP) have come under much scrutiny. However, the lack of experimental evidence, combined with non-random selection into these programs, makes identification of the causal effects of such programs difficult. In the case of the SBP, this difficulty is exacerbated by the apparent lack of exclusion restrictions. Here, we compare via Monte Carlo study several existing estimators that do not rely on exclusion restrictions for identification. In addition, we propose two new estimation strategies. Simulations illustrate the usefulness of our new estimators, as well as provide applied researchers several practical guidelines when analyzing the causal effects of binary treatments. More importantly, we find consistent evidence of a beneficial causal effect of SBP participation on childhood obesity when applying estimators designed to circumvent selection on unobservables.

[1]  Daniel L. Millimet,et al.  On the Specification of Propensity Scores, With Applications to the Analysis of Trade Policies , 2009 .

[2]  Justin McCrary,et al.  New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators , 2009, SSRN Electronic Journal.

[3]  J. Angrist,et al.  Identification and Estimation of Local Average Treatment Effects , 1995 .

[4]  T. Byers,et al.  Do obese children become obese adults? A review of the literature. , 1993, Preventive medicine.

[5]  Patrick Royston,et al.  Comment on sg3.4 and an Improved D'Agostino Test , 1992 .

[6]  Daniel L. Millimet,et al.  School Nutrition Programs and the Incidence of Childhood Obesity , 2008 .

[7]  O. Rosin The Economic Causes of Obesity: A Survey , 2008 .

[8]  Shihti Yu,et al.  Collinearity and Two-Step Estimation of Sample Selection Models:Problems, Origins, and Remedies , 2000 .

[9]  J. Heckman,et al.  Longitudinal Analysis of Labor Market Data: Alternative methods for evaluating the impact of interventions , 1985 .

[10]  J. Heckman,et al.  The Economics and Econometrics of Active Labor Market Programs , 1999 .

[11]  G. Imbens,et al.  Estimation of Causal Effects using Propensity Score Weighting: An Application to Data on Right Heart Catheterization , 2001, Health Services and Outcomes Research Methodology.

[12]  Alyce Adams,et al.  Why is the Developed World Obese? , 2007, Annual review of public health.

[13]  G. Imbens,et al.  Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score , 2000 .

[14]  R. Rohh ALTERNATIVE METHODS FOR EVALUATING THE IMPACT OF INTERVENTIONS An Overview , 2001 .

[15]  G. Imbens,et al.  Matching on the Estimated Propensity Score , 2009 .

[16]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[17]  Michael Weitzman,et al.  Effects of childhood obesity on hospital care and costs, 1999-2005. , 2009, Health affairs.

[18]  Lung-fei Lee,et al.  Tests for the Bivariate Normal Distribution in Econometric Models with Selectivity , 1984 .

[19]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[20]  Todd C. Headrick,et al.  Simulating correlated multivariate nonnormal distributions: Extending the fleishman power method , 1999 .

[21]  Richard K. Crump,et al.  Dealing with limited overlap in estimation of average treatment effects , 2009 .

[22]  T. Speed,et al.  On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9 , 1990 .

[23]  D. Rubin Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[24]  David S. Lee,et al.  Regression Discontinuity Designs in Economics , 2009 .

[25]  M. Lechner,et al.  How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score , 2010, SSRN Electronic Journal.

[26]  Does the National School Lunch Program Improve Children's Dietary Outcomes? , 2011 .

[27]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[28]  R. A. Fisher,et al.  Design of Experiments , 1936 .

[29]  M. Berger,et al.  Bounding Parameter Estimates with Nonclassical Measurement Error , 2000 .

[30]  L. Janelli Body Image in Older Adults: A Review of the Literature , 1986, Rehabilitation nursing : the official journal of the Association of Rehabilitation Nurses.

[31]  F. Vella,et al.  A Semiparametric Model for Binary Response and Continuous Outcomes Under Index Heteroscedasticity , 2006 .

[32]  Arthur S. Goldberger,et al.  ABNORMAL SELECTION BIAS , 1983 .

[33]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[34]  Patrick A. Puhani,et al.  The Heckman Correction for Sample Selection and Its Critique - A Short Survey , 2000 .

[35]  D. Black,et al.  How robust is the evidence on the effects of college quality? Evidence from matching , 2004 .

[36]  R. D'Agostino Adjustment Methods: Propensity Score Methods for Bias Reduction in the Comparison of a Treatment to a Non‐Randomized Control Group , 2005 .

[37]  Jonathan H. Wright,et al.  A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments , 2002 .

[38]  D. Horvitz,et al.  A Generalization of Sampling Without Replacement from a Finite Universe , 1952 .

[39]  K. Hupfeld,et al.  Program evaluation. , 1989, NLN publications.

[40]  A. Smithb,et al.  How robust is the evidence on the e ects of college quality ? Evidence from matching , 2004 .

[41]  Steven J. Haider,et al.  Breakfast of Champions? , 2006, The Journal of Human Resources.

[42]  Kanti V. Mardia,et al.  Families of Bivariate Distributions , 1970 .

[43]  R. D'Agostino Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. , 2005, Statistics in medicine.

[44]  D. Rubin [On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.] Comment: Neyman (1923) and Causal Inference in Experiments and Observational Studies , 1990 .

[45]  Shakeeb Khan,et al.  Irregular Identification, Support Conditions, and Inverse Weight Estimation , 2010 .

[46]  D. Schanzenbach Do School Lunches Contribute to Childhood Obesity? , 2009, The Journal of Human Resources.

[47]  Daniel L. Millimet,et al.  The Elephant in the Corner: A Cautionary Tale About Measurement Error in Treatment Effects Models , 2010, SSRN Electronic Journal.

[48]  James J Heckman,et al.  Understanding Instrumental Variables in Models with Essential Heterogeneity , 2006, The Review of Economics and Statistics.

[49]  H. Peña The Control Function , 1974 .

[50]  Bar-Ilan THE ECONOMIC CAUSES OF OBESITY : A SURVEY , 2008 .

[51]  Michel Wedel,et al.  Frugal IV alternatives to identify the parameter for an endogenous regressor , 2009 .

[52]  T. Mroz,et al.  Discrete factor approximations in simultaneous equation models: estimating the impact of a dummy endogenous variable on a continuous outcome. , 1999, Journal of econometrics.

[53]  Brian D. Johnson,et al.  Is the Magic Still There? The Use of the Heckman Two-Step Correction for Selection Bias in Criminology , 2007 .

[54]  James J. Heckman,et al.  Longitudinal Analysis of Labor Market Data , 1985 .

[55]  Christopher R. Taber,et al.  Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools , 2000, Journal of Political Economy.

[56]  E. Finkelstein,et al.  Annual medical spending attributable to obesity: payer-and service-specific estimates. , 2009, Health affairs.