A Bayesian Selection Model for Correcting Outcome Reporting Bias With Application to a Meta-analysis on Heart Failure Interventions

Multivariate meta-analysis (MMA) is a powerful tool for jointly estimating multiple outcomes’ treatment effects. However, the validity of results from MMA is potentially compromised by outcome reporting bias (ORB), or the tendency for studies to selectively report outcomes. Until recently, ORB has been understudied. Since ORB can lead to biased conclusions, it is crucial to correct the estimates of effect sizes and quantify their uncertainty in the presence of ORB. With this goal, we develop a Bayesian selection model to adjust for ORB in MMA. We further propose a measure for quantifying the impact of ORB on the results from MMA. We evaluate our approaches through a meta-evaluation of 748 bivariate meta-analyses from the Cochrane Database of Systematic Reviews. Our model is motivated by and applied to a meta-analysis of interventions on hospital readmission and quality of life for heart failure patients. In our analysis, the relative risk (RR) of hospital readmission for the intervention group changes from a significant decrease (RR: 0.931, 95% confidence interval [CI]: 0.862-0.993) to a statistically nonsignificant effect (RR: 0.955, 95% CI: 0.876–1.051) after adjusting for ORB. This study demonstrates that failing to account for ORB can lead to different conclusions in a meta-analysis.

[1]  Richard D Riley,et al.  Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement , 2015 .

[2]  Richard D Riley,et al.  Multivariate meta‐analysis: the effect of ignoring within‐study correlation , 2009 .

[3]  C. Begg,et al.  Operating characteristics of a rank correlation test for publication bias. , 1994, Biometrics.

[4]  Rebecca S. Graves,et al.  Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. , 2002 .

[5]  J. Cleland,et al.  Structured telephone support or non-invasive telemonitoring for patients with heart failure , 2016, Heart.

[6]  G. Fonarow,et al.  The Prevention of Hospital Readmissions in Heart Failure. , 2016, Progress in cardiovascular diseases.

[7]  Richard D Riley,et al.  Beyond the Bench: Hunting Down Fugitive Literature , 2004, Environmental Health Perspectives.

[8]  Tianjing Li,et al.  Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. , 2017, Journal of clinical epidemiology.

[9]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[10]  I. Tannock,et al.  Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer. , 2013, Annals of oncology : official journal of the European Society for Medical Oncology.

[11]  H. Chu,et al.  Bayesian multivariate meta‐analysis of multiple factors , 2018, Research synthesis methods.

[12]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[13]  William C. Horrace,et al.  Some results on the multivariate truncated normal distribution , 2005 .

[14]  J. Copas,et al.  A model-based correction for outcome reporting bias in meta-analysis. , 2014, Biostatistics.

[15]  David Moher,et al.  Comparison of registered and published primary outcomes in randomized controlled trials. , 2009, JAMA.

[16]  D. Altman,et al.  Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research , 2004, Canadian Medical Association Journal.

[17]  Jing Ning,et al.  Maximum likelihood estimation and EM algorithm of Copas-like selection model for publication bias correction. , 2017, Biostatistics.

[18]  Richard D Riley,et al.  An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown. , 2008, Biostatistics.

[19]  S. Andreassen,et al.  Reporting of adverse events in randomized controlled trials of highly active antiretroviral therapy: systematic review. , 2009, The Journal of antimicrobial chemotherapy.

[20]  C. Celano,et al.  Depression and Anxiety in Heart Failure: A Review , 2018, Harvard review of psychiatry.

[21]  Dimitris Mavridis,et al.  A fully Bayesian application of the Copas selection model for publication bias extended to network meta‐analysis , 2013, Statistics in medicine.

[22]  D. Rennie,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative , 2003, BMJ : British Medical Journal.

[23]  J. Copas,et al.  Meta-analysis, funnel plots and sensitivity analysis. , 2000, Biostatistics.

[24]  J. Copas.  What works?: selectivity models and meta‐analysis , 1999 .

[25]  J Q Shi,et al.  A sensitivity analysis for publication bias in systematic reviews , 2001, Statistical methods in medical research.

[26]  T. Lumley.  Network meta‐analysis for indirect treatment comparisons , 2002, Statistics in medicine.

[27]  Haitao Chu,et al.  Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. , 2006, Journal of clinical epidemiology.

[28]  Richard D Riley,et al.  A multivariate meta‐analysis approach for reducing the impact of outcome reporting bias in systematic reviews , 2012, Statistics in medicine.

[29]  Dan Jackson,et al.  Multivariate meta-analysis: Potential and promise , 2011, Statistics in medicine.

[30]  Dan Jackson,et al.  Assessing the Implications of Publication Bias for Two Popular Estimates of between‐Study Variance in Meta‐Analysis , 2007, Biometrics.

[31]  P C Lambert,et al.  An evaluation of bivariate random‐effects meta‐analysis for the joint synthesis of two correlated outcomes , 2007, Statistics in medicine.

[32]  M. Pittler.  Systematic Reviews in Health Care: Meta‐analysis in Context , 2010 .

[33]  L. Hazell,et al.  Under-Reporting of Adverse Drug Reactions , 2006, Drug safety.

[34]  Lifeng Lin,et al.  Quantifying publication bias in meta‐analysis , 2018, Biometrics.

[35]  Alexander Tsertsvadze,et al.  Systematic reviews: when is an update an update? , 2006, The Lancet.

[36]  D. Altman,et al.  Chapter 8: Assessing risk of bias in included studies , 2008 .

[37]  P. Williamson,et al.  Multivariate meta-analysis helps examine the impact of outcome reporting bias in Cochrane rheumatoid arthritis reviews. , 2015, Journal of clinical epidemiology.

[38]  E. Hemminki.  Study of information submitted by drug companies to licensing authorities. , 1980, British medical journal.

[39]  S. Evans,et al.  Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet , 2008, The Lancet.

[40]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[41]  Yong Chen,et al.  A Robust Bayesian Copas Selection Model for Quantifying and Correcting Publication Bias , 2020, 2005.02930.

[42]  A. Hrõbjartsson,et al.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. , 2004, JAMA.

[43]  Bennett T. Shamsai.  Users' guides to the medical literature: A manual for evidence-based clinical practice , 2002 .

[44]  I. Olkin,et al.  Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement , 1999, The Lancet.

[45]  J. Sterne,et al.  The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.