Propensity Score Matching and Policy Impact Analysis: A Demonstration in Eviews

Effective development policymaking creates a need for reliable methods of assessing effectiveness. There should be, therefore, an intimate relationship between effective policymaking and impact analysis. The goal of a development intervention defines the metric by which to assess its impact, while impact evaluation can produce reliable information on which policymakers may base decisions to modify or cancel ineffective programs and thus make the most of limited resources. This paper reviews the logic of propensity score matching (PSM) and, using data on the National Support Work Demonstration, compares that approach with other evaluation methods such as double difference, instrumental variable, and Heckman's method of selection bias correction. In addition, it demonstrates how to implement nearest-neighbor and kernel-based methods, and plot program incidence curves in E-Views. In the end, the plausibility of an evaluation method hinges critically on the correctness of the socioeconomic model underlying program design and implementation, and on the quality and quantity of available data. In any case, PSM can act as an effective adjuvant to other methods.

[1]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[2]  M. Ravallion,et al.  Estimating the Benefit Incidence of an Antipoverty Program by Propensity-Score Matching , 2003 .

[3]  R. Lalonde Evaluating the Econometric Evaluations of Training Programs with Experimental Data , 1984 .

[4]  P. Schmidt,et al.  Limited-Dependent and Qualitative Variables in Econometrics. , 1984 .

[5]  J. Heckman,et al.  Removing the Veil of Ignorance in Assessing the Distributional Impacts of Social Policies , 2002, SSRN Electronic Journal.

[6]  W. Cleveland,et al.  Smoothing by Local Regression: Principles and Methods , 1996 .

[7]  Shaohua Chen,et al.  Measuring Pro-Poor Growth , 1999 .

[8]  Jeffrey M. Woodbridge Econometric Analysis of Cross Section and Panel Data , 2002 .

[9]  Marco Caliendo,et al.  Some Practical Guidance for the Implementation of Propensity Score Matching , 2005, SSRN Electronic Journal.

[10]  Martin Ravallion,et al.  Hidden Impact? Ex-Post Evaluation of an Anti-Poverty Program , 2003 .

[11]  Sascha O. Becker,et al.  Estimation of Average Treatment Effects Based on Propensity Scores , 2002 .

[12]  B. Sianesi Implementing Propensity Score Matching Estimators with STATA , 2001 .

[13]  M. Ravallion The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation , 2001 .

[14]  J. Grossman,et al.  Evaluating social policies : principles and U.S. experience , 1994 .

[15]  J. Baker Evaluating the Impact of Development Projects on Poverty: A Handbook for Practitioners , 2000 .

[16]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[17]  S. Paternostro,et al.  The Returns to Participation in the Nonfarm Sector in Rural Rwanda , 2004 .

[18]  R. Blundell,et al.  Evaluation Methods for Non‐Experimental Data , 2005 .

[19]  Rajeev Dehejia Practical propensity score matching: a reply to Smith and Todd , 2005 .

[20]  Alternative approaches to evaluation in empirical microeconomics , 2002 .

[21]  Petra E. Todd,et al.  Reconciling Conflicting Evidence on the Performance of Propensity-Score Matching Methods , 2001 .

[22]  J. Heckman The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models , 1976 .

[23]  C. Glymour,et al.  STATISTICS AND CAUSAL INFERENCE , 1985 .

[24]  Evaluating the Impact of Infrastructure Rehabilitation Projects on Household Welfare in Rural Georgia , 2003 .

[25]  Martin Ravallion,et al.  Hidden impact? Household saving in response to a poor-area development project , 2005 .

[26]  N. Monteiro Using propensity matching estimators to evaluate the impact of privatization on wages , 2004 .

[27]  Marco Caliendo,et al.  The microeconometric estimation of treatment effects—An overview , 2005 .

[28]  Michael Lokshin,et al.  Maximum Likelihood Estimation of Endogenous Switching Regression Models , 2004 .

[29]  R. Stott,et al.  The World Bank , 2008, Annals of tropical medicine and parasitology.

[30]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[31]  Rajeev Dehejia,et al.  Propensity Score-Matching Methods for Nonexperimental Causal Studies , 2002, Review of Economics and Statistics.

[32]  G. Patil,et al.  Rejoinder , 2004, Environmental and Ecological Statistics.

[33]  W. Cleveland Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[34]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[35]  Lung-fei Lee Generalized Econometric Models with Selectivity , 1983 .

[36]  M. Ravallion Evaluating Anti-Poverty Programs , 2005 .

[37]  Introduction to the Symposium on the Econometrics of Matching , 2004, Review of Economics and Statistics.

[38]  M. Ravallion,et al.  Gainers and Losers from Trade Reform in Morocco , 2004 .

[39]  James J. Heckman,et al.  Assessing the Case for Social Experiments , 1995 .

[40]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .

[41]  B. Essama-Nssah Empowerment and poverty-focused evaluation , 2004 .

[42]  M. Ravallion Assessing the Poverty Impact of an Assigned Program , 2022 .