Drawing Causal Inferences Using Propensity Scores: A Practical Guide for Community Psychologists

Confounding present in observational data impede community psychologists’ ability to draw causal inferences. This paper describes propensity score methods as a conceptually straightforward approach to drawing causal inferences from observational data. A step-by-step demonstration of three propensity score methods—weighting, matching, and subclassification—is presented in the context of an empirical examination of the causal effect of preschool experiences (Head Start vs. parental care) on reading development in kindergarten. Although the unadjusted population estimate indicated that children with parental care had substantially higher reading scores than children who attended Head Start, all propensity score adjustments reduce the size of this overall causal effect by more than half. The causal effect was also defined and estimated among children who attended Head Start. Results provide no evidence for improved reading if those children had instead received parental care. We carefully define different causal effects and discuss their respective policy implications, summarize advantages and limitations of each propensity score method, and provide SAS and R syntax so that community psychologists may conduct causal inference in their own research.

[1]  T. VanderWeele The use of propensity score methods in psychiatric research , 2006, International journal of methods in psychiatric research.

[2]  L. Green,et al.  Limitations of the randomized controlled trial in evaluating population-based health interventions. , 2007, American journal of preventive medicine.

[3]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data: Little/Statistical Analysis with Missing Data , 2002 .

[4]  P. Austin An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies , 2011, Multivariate behavioral research.

[5]  S. West Alternatives to Randomized Experiments , 2009 .

[6]  Elizabeth,et al.  Matching Methods for Causal Inference , 2007 .

[7]  P. Rosenbaum The Consequences of Adjustment for a Concomitant Variable that Has Been Affected by the Treatment , 1984 .

[8]  G. Imbens,et al.  Estimation of Causal Effects using Propensity Score Weighting: An Application to Data on Right Heart Catheterization , 2001, Health Services and Outcomes Research Methodology.

[9]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[10]  R. D'Agostino Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. , 2005, Statistics in medicine.

[11]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[12]  Peter C. Austin,et al.  A Tutorial and Case Study in Propensity Score Analysis: An Application to Estimating the Effect of In-Hospital Smoking Cessation Counseling on Mortality , 2011, Multivariate behavioral research.

[13]  D. Altman,et al.  Missing data , 2007, BMJ : British Medical Journal.

[14]  Elizabeth A Stuart,et al.  Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. , 2010, Psychological methods.

[15]  Joseph L Schafer,et al.  Analysis of Incomplete Multivariate Data , 1997 .

[16]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[17]  Gary King,et al.  Misunderstandings between experimentalists and observationalists about causal inference , 2008 .

[18]  J. Schafer,et al.  Average causal effects from nonrandomized studies: a practical guide and simulated example. , 2008, Psychological methods.

[19]  J. Culpepper National Center for Educational Statistics , 2007 .

[20]  Gary King,et al.  The Dangers of Extreme Counterfactuals , 2006, Political Analysis.

[21]  William R Shadish,et al.  Propensity Scores , 2005, Evaluation review.

[22]  D. Rubin,et al.  Statistical Analysis with Missing Data , 1988 .

[23]  J. Avorn,et al.  Variable selection for propensity score models. , 2006, American journal of epidemiology.

[24]  Thomas D. Cook,et al.  Why have Educational Evaluators Chosen Not to Do Randomized Experiments? , 2003 .

[25]  G. W. Imbens Sensitivity to Exogeneity Assumptions in Program Evaluation , 2003 .

[26]  Elizabeth A Stuart,et al.  Improving propensity score weighting using machine learning , 2010, Statistics in medicine.

[27]  Thanh Le,et al.  Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K): Combined User's Manual for the ECLS-K Eighth-Grade and K-8 Full Sample Data Files and Electronic Codebooks. NCES 2009-004. , 2009 .

[28]  R. MacCoun Experimental and Quasi‐Experimental Designs for Generalized Causal Inference, by William R. Shadish, Thomas D. Cook, and Donald T. Campbell. Boston: Houghton Mifflin, 2001, 623 pp., $65.56. , 2003 .

[29]  Frank J. Potter,et al.  THE EFFECT OF WEIGHT TRIMMING ON NONLINEAR SURVEY ESTIMATES , 2002 .

[30]  D. McCaffrey,et al.  Propensity score estimation with boosted regression for evaluating causal effects in observational studies. , 2004, Psychological methods.

[31]  Judea Pearl,et al.  On a Class of Bias-Amplifying Variables that Endanger Effect Estimates , 2010, UAI.

[32]  Shawna L. Mercer,et al.  Study designs for effectiveness and translation research :identifying trade-offs. , 2007, American journal of preventive medicine.

[33]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[34]  E. Stuart,et al.  Estimating Intervention Effects of Prevention Programs: Accounting for Noncompliance , 2008, Prevention Science.

[35]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[36]  E. Stuart,et al.  Using full matching to estimate causal effects in nonexperimental studies: examining the relationship between adolescent marijuana use and adult outcomes. , 2008, Developmental psychology.

[37]  Peter M. Steiner,et al.  The importance of covariate selection in controlling for selection bias in observational studies. , 2010, Psychological methods.

[38]  R. Mccall,et al.  Beyond the Methodological Gold Standards of Behavioral Research: Considerations for Practice and Policy , 2004 .

[39]  M. Greenberg,et al.  Examining the link between preschool social-emotional competence and first grade academic achievement: The role of attention skills , 2011 .