A Note on the Common Support Problem in Applied Evaluation Studies

This paper advocates the use of a nonparametric bounds analysis to check the robustness of the results of applied evaluation studies to the problem of a lack of common support. The typical responses by researchers of either ignoring it, or obtaining estimates only for the subpopulation within the common support, can both be misleading: Ignoring the problem may result in biases because the comparison group may not be comparable. Deleting observations at best yields an estimator that is consistent for the common support only. When treatment effects differ inside and outside the common support this estimator is inconsistent for the parameter of interest. Furthermore, useful information is ignored, because, for example, the response of the treated to the treatment can be estimated even outside the common support. This information can be used to derive bounds with widths depending on the configuration of the data. The application to an evaluation study of Swiss active labour market policies shows that the relevance of the bounds for changing the interpretation of the results depends very much on the particular data configuration.

[1]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[2]  G. Imbens,et al.  Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score , 2000 .

[3]  M. Lechner,et al.  A Microeconometric Evaluation of Active Labor Market Policy in Switzerland , 2001 .

[4]  J. Hahn.  On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects , 1998 .

[5]  M. Lechner.  An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany , 1999 .

[6]  C. Manski.  Nonparametric Bounds on Treatment Effects , 1989 .

[7]  Michael Lechner,et al.  Microeconometric Evaluation of the Active Labour Market Policy in Switzerland , 2000 .

[8]  D. Rubin.  ASSIGNMENT TO TREATMENT GROUP ON THE BASIS OF A COVARIATE , 1976 .

[9]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[10]  James J. Heckman,et al.  Alternative methods for solving the problem of selection bias in evaluating the impact of treatments , 1986 .

[11]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[12]  C. Manski.  Anatomy of the Selection Problem , 1989 .

[13]  M. Lechner.  Identification and Estimation of Causal Effects of Multiple Treatments Under the Conditional Independence Assumption , 1999, SSRN Electronic Journal.

[14]  J. Angrist,et al.  Identification and Estimation of Local Average Treatment Effects , 1994 .

[15]  J. Heckman,et al.  The Economics and Econometrics of Active Labor Market Programs , 1999 .

[16]  M. Lechner.  Earnings and Employment Effects of Continuous Gff-the-Job Training in East Germany After Unification , 1995 .

[17]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[18]  Tobias Hagen.  Do Temporary Workers Receive Risk Premiums? Assessing the Wage Effects of Fixed-Term Contracts in West Germany by a Matching Estimator Compared with Parametric Approaches , 2001 .

[19]  Michael Lechner,et al.  Nonparametric bounds on employment and income effects of continuous vocational training in East Germany , 1999 .

[20]  J. Angrist,et al.  Identification and Estimation of Local Average Treatment Effects , 1995 .

[21]  J J Heckman,et al.  Sources of selection bias in evaluating social programs: an interpretation of conventional measures and evidence on the effectiveness of matching as a program evaluation method. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[22]  G. Imbens.  The Role of the Propensity Score in Estimating Dose-Response Functions , 1999 .

[23]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .