Are propensity scores really superior to standard multivariable analysis?

Clinicians often face difficult decisions despite the lack of evidence from randomized trials. Thus, clinical evidence is often shaped by non-randomized studies exploiting multivariable approaches to limit the extent of confounding. Since their introduction, propensity scores have been used more and more frequently to estimate relevant clinical effects adjusting for established confounders, especially in small datasets. However, debate persists on their real usefulness in comparison to standard multivariable approaches such as logistic regression and Cox proportional hazard analysis. This holds even truer in light of key quantitative developments such as bootstrap and Bayesian methods. This qualitative review aims to provide a concise and practical guide to choose between propensity scores and standard multivariable analysis, emphasizing strengths and weaknesses of both approaches.

[1]  M Alan Brookhart,et al.  American Journal of Epidemiology Practice of Epidemiology Instrumental Variable Analysis for Estimation of Treatment Effects with Dichotomous Outcomes , 2022 .

[2]  P. Serruys,et al.  Comparison of early outcome of percutaneous coronary intervention for unprotected left main coronary artery disease in the drug-eluting stent era with versus without intravascular ultrasonic guidance. , 2005, The American journal of cardiology.

[3]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[4]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[5]  Peter C Austin,et al.  Report Card on Propensity-Score Matching in the Cardiology Literature From 2004 to 2006: A Systematic Review , 2008, Circulation. Cardiovascular quality and outcomes.

[6]  D. Capodanno,et al.  Routine versus selective coronary artery bypass for left main coronary artery revascularization: the Appraise a Customized Strategy for Left Main Revascularization (CUSTOMIZE) study. , 2011, International journal of cardiology.

[7]  James O. Berger,et al.  The interplay of Bayesian and frequentist analysis , 2004 .

[8]  Gordon H. Guyatt,et al.  Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice , 2003, BMJ : British Medical Journal.

[9]  M. Hadamitzky,et al.  Predictive factors of restenosis after coronary stent placement. , 1997, Journal of the American College of Cardiology.

[10]  Peter C Austin,et al.  The performance of different propensity-score methods for estimating relative risks. , 2008, Journal of clinical epidemiology.

[11]  N. Jewell,et al.  Some surprising results about covariate adjustment in logistic regression models , 1991 .

[12]  Peter C Austin,et al.  Some Methods of Propensity‐Score Matching had Superior Performance to Others: Results of an Empirical Investigation and Monte Carlo simulations , 2009, Biometrical journal. Biometrische Zeitschrift.

[13]  I. Iakovou,et al.  Incidence, predictors, and outcomes of coronary dissections left untreated after drug-eluting stent implantation. , 2006, European heart journal.

[14]  D. Sackett Bias in analytic research. , 1979, Journal of chronic diseases.

[15]  S. de Servi,et al.  Real-world outcome of coronary bifurcation lesions in the drug-eluting stent era: results from the 4,314-patient Italian Society of Invasive Cardiology (SICI-GISE) Italian Multicenter Registry on Bifurcations (I-BIGIS). , 2010, American heart journal.

[16]  R B D'Agostino,et al.  Relation of pooled logistic regression to time dependent Cox regression analysis: the Framingham Heart Study. , 1990, Statistics in medicine.

[17]  E. Antman,et al.  An integrated clinical approach to predicting the benefit of tirofiban in non-ST elevation acute coronary syndromes. Application of the TIMI Risk Score for UA/NSTEMI in PRISM-PLUS. , 2002, European heart journal.

[18]  Elliott S Fisher,et al.  Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. , 2007, JAMA.

[19]  Marshall M Joffe,et al.  On the estimation and use of propensity scores in case-control and case-cohort studies. , 2007, American journal of epidemiology.

[20]  S. Kullander,et al.  HUMAN PLACENTAL LACTOGEN IN SCREENING FOR MULTIPLE PREGNANCIES , 1975, The Lancet.

[21]  J. Robins,et al.  Results of multivariable logistic regression, propensity matching, propensity adjustment, and propensity-based weighting under conditions of nonuniform effect. , 2006, American journal of epidemiology.

[22]  M Soledad Cepeda,et al.  Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. , 2003, American journal of epidemiology.

[23]  M. A. Best Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[24]  Peter C Austin,et al.  Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review. , 2005, Journal of clinical epidemiology.

[25]  Vincent Mor,et al.  Weaknesses of goodness‐of‐fit tests for evaluating propensity score models: the case of the omitted confounder , 2005, Pharmacoepidemiology and drug safety.

[26]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[27]  Peter C Austin,et al.  A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003 , 2008, Statistics in medicine.

[28]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[29]  R. D'Agostino Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. , 2005, Statistics in medicine.

[30]  P Peduzzi,et al.  Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. , 1995, Journal of clinical epidemiology.

[31]  W. Larrabee,et al.  Users' Guide to the Medical Literature: A Manual for Evidence-Based Clinical Practice , 2002 .

[32]  Anthonius de Boer,et al.  Systematic differences in treatment effect estimates between propensity score methods and logistic regression. , 2008, International journal of epidemiology.

[33]  I. Iakovou,et al.  Validation of predictors of intraprocedural stent thrombosis in the drug-eluting stent era. , 2005, The American journal of cardiology.

[34]  Til Stürmer,et al.  A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. , 2006, Journal of clinical epidemiology.

[35]  Peter C Austin,et al.  A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study , 2007, Statistics in medicine.

[36]  A. B. Hill The Environment and Disease: Association or Causation? , 1965, Proceedings of the Royal Society of Medicine.

[37]  P. Austin,et al.  The use of the propensity score for estimating treatment effects: administrative versus clinical data , 2005, Statistics in medicine.

[38]  D. Rubin,et al.  Combining Propensity Score Matching with Additional Adjustments for Prognostic Covariates , 2000 .

[39]  Nitin R. Patel,et al.  Exact logistic regression: theory and examples. , 1995, Statistics in medicine.

[40]  Vincent Mor,et al.  Principles for modeling propensity scores in medical research: a systematic literature review , 2004, Pharmacoepidemiology and drug safety.

[41]  Sunil J Rao,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2003 .

[42]  J. H. Mitchell Medical aid to vietnam. , 1969, The Medical journal of Australia.

[43]  Donald Rubin,et al.  Estimating Causal Effects from Large Data Sets Using Propensity Scores , 1997, Annals of Internal Medicine.

[44]  S. Burgess “Users’ Guide to the Medical Literature - A Manual for Evidence Based Clinical Practice”. Edited by Gordon Guyatt and Drummond Rennie, 2002, AMA Press, Chicago, 706 pages , 2003 .

[45]  R. Norris,et al.  A new coronary prognostic index. , 1970, Lancet.

[46]  M. Aldenderfer,et al.  Cluster Analysis. Sage University Paper Series On Quantitative Applications in the Social Sciences 07-044 , 1984 .

[47]  D. Cox,et al.  Analysis of Survival Data. , 1985 .

[48]  T. Kurth,et al.  Propensity scores: help or hype? , 2004, Nephrology, dialysis, transplantation : official publication of the European Dialysis and Transplant Association - European Renal Association.

[49]  Elizabeth A Stuart,et al.  Improving propensity score weighting using machine learning , 2010, Statistics in medicine.

[50]  J. Concato,et al.  Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. , 1995, Journal of clinical epidemiology.

[51]  C. Mooney,et al.  Monte Carlo Simulation , 1997 .

[52]  J. Concato,et al.  A simulation study of the number of events per variable in logistic regression analysis. , 1996, Journal of clinical epidemiology.

[53]  E W Steyerberg,et al.  Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. , 1999, Journal of clinical epidemiology.