Alternative approaches to evaluation in empirical microeconomics

Abstract.Four alternative but related approaches to empirical evaluation of policy interventions are studied: social experiments, natural experiments, matching methods, and instrumental variables. In each case the necessary assumptions and the data requirements are considered for estimation of a number of key parameters of interest. These key parameters include the average treatment effect, the treatment on the treated and the local average treatment effect. Some issues of implementation and interpretation are discussed drawing on the labour market programme evaluation literature.

[1]  C. Meghir,et al.  Education choices in Mexico: using a structural model and a randomized experiment to evaluate Progresa , 2010 .

[2]  Wilbert Van Der Klaauw,et al.  Regression-Discontinuity Analysis , 2010 .

[3]  W. V. D. Klaauw,et al.  Regression-Discontinuity Analysis: A Survey of Recent Developments in Economics , 2008 .

[4]  Costas Meghir,et al.  Identification of Treatment Effects Using Control Functions in Models with Continuous, Endogenous Treatment and Heterogeneous Effects , 2008 .

[5]  Erich Battistin,et al.  Ineligibles and eligible non-participants as a double comparison group in regression-discontinuity designs , 2008 .

[6]  Rob J. Hyndman,et al.  Selection of Models , 2008 .

[7]  S Lee,et al.  Changes in College Enrollment and Wage Inequality: Distinguishing Price and Composition Effects , 2008 .

[8]  A. C. van der Klauw,et al.  Nonparametric identification , 2020, Modeling, Identification and Simulation of Dynamical Systems.

[9]  R. Moffitt.  Estimating Marginal Treatment Effects in Heterogeneous Populations , 2008 .

[10]  S. Nomura.  [Primary care for depression]. , 2007, Nihon rinsho. Japanese journal of clinical medicine.

[11]  J. Heckman,et al.  Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation , 2007 .

[12]  Whitney Newey,et al.  Treatment Effects , 2007 .

[13]  Petra E. Todd,et al.  Assessing the Impact of a School Subsidy Program in Mexico: Using a Social Experiment to Validate a Dynamic Behavioral Model of Child Schooling and Fertility. , 2006, The American economic review.

[14]  M. Frölich.  A Note on Parametric and Nonparametric Regression in the Presence of Endogenous Control Variables , 2006 .

[15]  William R Shadish,et al.  Propensity Scores , 2005, Evaluation review.

[16]  G. Giorgi.  The New Deal for Young People Five Years On , 2005 .

[17]  David S. Lee.  Randomized experiments from non-random selection in U.S. House elections , 2005 .

[18]  R. Blundell,et al.  Evaluating the effect of education on earnings: models, methods and results from the National Child Development Survey , 2005 .

[19]  R. Blundell,et al.  Evaluation Methods for Non‐Experimental Data , 2005 .

[20]  C. Meghir,et al.  Education Choices in Mexico: Using a Structural Model and a Randomized Experiment to evaluate Progresa.∗ , 2005 .

[21]  J. Powell,et al.  Endogeneity in semiparametric binary response models , 2004 .

[22]  Michael R Elliott,et al.  Causal Models for Randomized Physician Encouragement Trials in Treating Primary Care Depression , 2004 .

[23]  James J. Heckman,et al.  Using matching, instrumental variables and control functions to estimate economic choice models , 2004 .

[24]  R. Koenker,et al.  Quantile regression methods for recursive structural equation models , 2004 .

[25]  B. Sianesi.  An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s , 2004, Review of Economics and Statistics.

[26]  Ian Hudson,et al.  Removing the Veil? , 2003 .

[27]  Laura Larsson.  Evaluation of Swedish Youth Labor Market Programs , 2003, The Journal of Human Resources.

[28]  Gerard J. van den Berg,et al.  The nonparametric identification of treatment effects in duration models , 2003 .

[29]  A. Chesher.  Identification in Nonseparable Models , 2003 .

[30]  J. Heckman,et al.  Estimating distributions of counterfactuals with an application to the returns to schooling and measurement of the effect of uncertainty on schooling choice , 2003 .

[31]  J. Heckman,et al.  Estimating Distributions of Treatment Effects with an Application to the Returns to Schooling and Measurement of the Effects of Uncertainty on College Choice , 2003, SSRN Electronic Journal.

[32]  Whitney K. Newey,et al.  Nonparametric Estimation of Sample Selection Models , 2003 .

[33]  B. Sianesi,et al.  PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing , 2003 .

[34]  Lars Peter Hansen,et al.  Advances in Economics and Econometrics , 2003 .

[35]  Sascha O. Becker,et al.  Estimation of Average Treatment Effects Based on Propensity Scores , 2002 .

[36]  G. Imbens,et al.  Identification and Estimation of Triangular Simultaneous Equations Models without Additivity , 2002 .

[37]  Jean-Pierre Florens,et al.  Instrumental variables, local instrumental variables and control functions , 2002 .

[38]  J. Heckman,et al.  Removing the Veil of Ignorance in Assessing the Distributional Impacts of Social Policies , 2002, SSRN Electronic Journal.

[39]  E. Vytlacil.  Independence, Monotonicity, and Latent Index Models: An Equivalence Result , 2002 .

[40]  Richard Blundell,et al.  Endogeneity in Nonparametric and Semiparametric Regression Models , 2022 .

[41]  Exogenous impact and conditional quantile functions , 2001 .

[42]  Costas Meghir,et al.  Evaluating the Employment Impact of a Mandatory Job Search Assistance Program , 2001 .

[43]  G. Loftus.  OF EXPERIMENTAL DATA , 2001 .

[44]  R. Rohh.  ALTERNATIVE METHODS FOR EVALUATING THE IMPACT OF INTERVENTIONS An Overview , 2001 .

[45]  J. Hahn,et al.  IDENTIFICATION AND ESTIMATION OF TREATMENT EFFECTS WITH A REGRESSION-DISCONTINUITY DESIGN , 2001 .

[46]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[47]  Edward Vytlacil,et al.  Local Instrumental Variables , 2000 .

[48]  Austan Goolsbee.  What Happens When You Tax the Rich? Evidence from Executive Compensation , 1997, Journal of Political Economy.

[49]  J. Angrist,et al.  Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings , 1999 .

[50]  J. Heckman,et al.  The Pre‐programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies , 1999 .

[51]  W. Newey,et al.  Nonparametric estimation of triangular simultaneous equations models , 1999 .

[52]  J J Heckman,et al.  Local instrumental variables and latent variable models for identifying and bounding treatment effects. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[53]  Jeffrey A. Smith,et al.  The Pre-Program Earnings Dip and the Determinants of Participation in a Social Program: Implications for Simple Program Evaluation Strategies , 1999 .

[54]  J. Heckman,et al.  The Economics and Econometrics of Active Labor Market Programs , 1999 .

[55]  R. Blundell,et al.  Labor Supply: A Review of Alternative Approaches , 1998 .

[56]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[57]  Donald W. K. Andrews,et al.  Semiparametric Estimation of the Intercept of a Sample Selection Model , 1998 .

[58]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[59]  Alberto Abadie,et al.  Instrumental Variables Estimation of Quantile Treatment Effects , 1998 .

[60]  J. Hahn.  On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects , 1998 .

[61]  James J. Heckman,et al.  Instrumental Variables Methods for the Correlated Random Coefficient Model: Estimating the Average Rate of Return to Schooling When the Return is Correlated with Schooling , 1998 .

[62]  J. Heckman,et al.  Making the Most out of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in Programme Impacts , 1997 .

[63]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[64]  J. Heckman.  Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations. , 1997 .

[65]  Costas Meghir,et al.  The determinants and effects of work-related training in Britain , 1996 .

[66]  David Card,et al.  Do Financial Incentives Encourage Welfare Recipients to Work? Early Findings from the Canadian Self Sufficiency Project , 1996 .

[67]  James J. Heckman,et al.  Randomization as an Instrumental Variable , 1995 .

[68]  Costas Meghir,et al.  Estimating labour supply responses using tax policy reforms , 1995 .

[69]  Jeffrey B. Liebman,et al.  Labor Supply Response to the Earned Income Tax Credit , 1995 .

[70]  James J. Heckman,et al.  The structure and consequences of eligibility rules for a social program: a study of the Job Training Partnership Act (JTPA). , 1995 .

[71]  James L. Powell,et al.  Estimation of semiparametric models , 1994 .

[72]  J. Powell,et al.  Semiparametric estimation of censored selection models with a nonparametric selection mechanism , 1993 .

[73]  J. Kemple.  The National JTPA Study:: Site Characteristics and Participation Patterns , 1993 .

[74]  J. Poterba,et al.  Income Inequality and the Incomes of Very High-Income Taxpayers: Evidence from Tax Returns , 1992, Tax Policy and the Economy.

[75]  H. London,et al.  Social experimentation , 1993 .

[76]  J. Angrist,et al.  Identification and Estimation of Local Average Treatment Effects , 1994 .

[77]  Jianqing Fan.  Design-adaptive Nonparametric Regression , 1992 .

[78]  H. James.  VARIETIES OF SELECTION BIAS , 1990 .

[79]  Lawrence H. Summers,et al.  Tax policy and the economy , 1989 .

[80]  James J. Heckman,et al.  Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: the Case of Manpower Training , 1989 .

[81]  M. Nerlove,et al.  Biases in dynamic models with fixed effects , 1988 .

[82]  Lawrence B. Lindsey.  Individual taxpayer response to tax cuts: 1982-1984: With implications for the revenue maximizing tax rate , 1987 .

[83]  Robert A. Moffitt,et al.  The Estimation of Wage Gains and Welfare Gains in Self-selection , 1987 .

[84]  Howard Wainer,et al.  Drawing inferences from self-selected samples , 1986 .

[85]  James J. Heckman,et al.  Alternative methods for solving the problem of selection bias in evaluating the impact of treatments , 1986 .

[86]  Gary Burtless,et al.  Are Targeted Wage Subsidies Harmful? Evidence from a Wage Voucher Experiment , 1985 .

[87]  J. Heckman,et al.  Longitudinal Analysis of Labor Market Data: Alternative methods for evaluating the impact of interventions , 1985 .

[88]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[89]  Orley Ashenfelter,et al.  Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs , 1984 .

[90]  R. Lalonde.  Evaluating the Econometric Evaluations of Training Programs with Experimental Data , 1984 .

[91]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[92]  Laurie J. Bassi,et al.  Estimating the Effect of Training Programs with Non-Random Selection , 1984 .

[93]  Anders Björklund,et al.  Estimation of Wage Gains and Welfare Gains from Self-Selection Models , 1983 .

[94]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[95]  Laurie J. Bassi.  THE EFFECT OF CETA ON THE POSTPROGRAM EARNINGS OF PARTICIPANTS , 1983 .

[96]  J. Heckman.  Sample selection bias as a specification error , 1979 .

[97]  D. Rubin,et al.  Using Multivariate Matched Sampling and Regression Adjustment to Control Bias in Observational Studies , 1978 .

[98]  Donald B. Rubin,et al.  Bayesian Inference for Causal Effects: The Role of Randomization , 1978 .

[99]  O. Ashenfelter,et al.  Estimating the Effect of Training Programs on Earnings , 1976 .

[100]  J. Heckman.  The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models , 1976 .

[101]  W. G. Cochran,et al.  Controlling Bias in Observational Studies: A Review. , 1974 .

[102]  R. Gronau.  Wage Comparisons--A Selectivity Bias , 1973, Journal of Political Economy.

[103]  R Fisher,et al.  Design of Experiments , 1936 .