Using Propensity Scores to Construct Comparable Control Groups for Disease Management Program Evaluation

IntroductionThe ability of observational studies to draw conclusions on causal relationships between covariates and outcomes can be improved by incorporating randomly matched controls using the propensity scoring method. This procedure controls for pre-program differences between the enrolled and non-enrolled groups by reducing each participant’s set of covariates into a single score, which makes it feasible to match on what are essentially multiple variables simultaneously. This paper introduces this concept using the first year results of a congestive heart failure (CHF) disease management (DM) program as an example.MethodsThis study employed a case-control pre-post study design with controls randomly matched to patients based on the propensity score. There were 94 patients with CHF enrolled in a DM program for at least 1 year (cases), who were matched to 94 patients with CHF drawn from a health plan’s CHF population (controls). Independent variables that estimated the propensity score were pre-program: hospital admissions, emergency department (ED) visits, total costs, and risk level. Baseline (1 year prior to program commencement) and 1-year outcome variables were compared for the two groups.ResultsThe results indicated that, at post-program, program participants had significantly lower hospitalization rates (p = 0.005), ED visit rates (p = 0.048), and total costs (p = 0.003) than their matched controls drawn from the CHF population.ConclusionsBecause of its simplicity and utility, propensity scoring should be considered as an alternative procedure for use with current non-experimental designs in evaluating DM program effectiveness.

[1]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[2]  L L Roos,et al.  Comparing clinical information with claims data: some similarities and differences. , 1991, Journal of clinical epidemiology.

[3]  P. Rosenbaum.  Sensitivity analysis for certain permutation inferences in matched observational studies , 1987 .

[4]  David R. Cox.  The analysis of binary data , 1970 .

[5]  D. Mark,et al.  Bias in the coding of hospital discharge data and its implications for quality assessment. , 1994, Medical care.

[6]  R. Andersen.  A behavioral model of families' use of health services , 1968 .

[7]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[8]  D B Rubin,et al.  Matching using estimated propensity scores: relating theory to practice. , 1996, Biometrics.

[9]  Michael Clarke,et al.  Non-random Reflections on Health Services Research , 1998, BMJ.

[10]  Evaluating Disease Management Program Effectiveness Adjusting for Enrollment (Tenure) and Seasonality , 2004 .

[11]  C. Drake.  Effects of misspecification of the propensity score on estimators of treatment effect , 1993 .

[12]  Donald Rubin,et al.  Estimating Causal Effects from Large Data Sets Using Propensity Scores , 1997, Annals of Internal Medicine.

[13]  E S Fisher,et al.  Overcoming potential pitfalls in the use of Medicare data for epidemiologic research. , 1990, American journal of public health.

[14]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[15]  J. Adams,et al.  An assessment of the total population approach for evaluating disease management program effectiveness. , 2003, Disease management : DM.

[16]  D. Campbell,et al.  EXPERIMENTAL AND QUASI-EXPERIMENT Al DESIGNS FOR RESEARCH , 2012 .

[17]  D. Rubin,et al.  Reducing Bias in Observational Studies Using Subclassification on the Propensity Score , 1984 .

[18]  P. Canner.  How much data should be collected in a clinical trial? Experience of the coronary drug project. , 1984, Statistics in medicine.

[19]  P. Canner,et al.  Covariate adjustment of treatment effects in clinical trials. , 1991, Controlled clinical trials.

[20]  Early controlled clinical trials. , 1996 .

[21]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[22]  Ariel Linden,et al.  Evaluating disease management program effectiveness: an introduction to time-series analysis. , 2003, Disease management : DM.

[23]  R. Andersen,et al.  Access to Medical Care in the U.S: Realized and Potential , 1978, Medical care.

[24]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[25]  D. Rubin.  ASSIGNMENT TO TREATMENT GROUP ON THE BASIS OF A COVARIATE , 1976 .

[26]  J. Adams,et al.  Evaluating disease management program effectiveness: an introduction to survival analysis. , 2004, Disease management : DM.

[27]  R. Andersen,et al.  Equity of Access to Medical Care: A Conceptual and Empirical Overview , 1981, Medical care.

[28]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[29]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[30]  D. Cox.  The Analysis of Multivariate Binary Data , 1972 .

[31]  W. G. Cochran.  The effectiveness of adjustment by subclassification in removing bias in observational studies. , 1968, Biometrics.