The Quality of Reporting Methods and Results in Network Meta-Analyses: An Overview of Reviews and Suggestions for Improvement

Introduction Some have suggested the quality of reporting of network meta-analyses (a technique used to synthesize information to compare multiple interventions) is sub-optimal. We sought to review information addressing this claim. Objective To conduct an overview of existing evaluations of quality of reporting in network meta-analyses and indirect treatment comparisons, and to compile a list of topics which may require detailed reporting guidance to enhance future reporting quality. Methods An electronic search of Medline and the Cochrane Registry of methodologic studies (January 2004–August 2013) was performed by an information specialist. Studies describing findings from quality of reporting assessments were sought. Screening of abstracts and full texts was performed by two team members. Descriptors related to all aspects of reporting a network meta-analysis were summarized. Results We included eight reports exploring the quality of reporting of network meta-analyses. From past reviews, authors found several aspects of network meta-analyses were inadequately reported, including primary information about literature searching, study selection, and risk of bias evaluations; statement of the underlying assumptions for network meta-analysis, as well as efforts to verify their validity; details of statistical models used for analyses (including information for both Bayesian and Frequentist approaches); completeness of reporting of findings; and approaches for summarizing probability measures as additional important considerations. Conclusions While few studies were identified, several deficiencies in the current reporting of network meta-analyses were observed. These findings reinforce the need to develop reporting guidance for network meta-analyses. Findings from this review will be used to guide next steps in the development of reporting guidance for network meta-analysis in the format of an extension of the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) Statement.

[1]  Giuseppe Biondi-Zoccai,et al.  A Bayesian network meta-analysis for binary outcome: how to do it , 2016, Statistical methods in medical research.

[2]  Andrew W Lee,et al.  Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. , 2014, Journal of clinical epidemiology.

[3]  Christopher H. Schmid,et al.  Characteristics of Networks of Interventions: A Description of a Database of 186 Published Networks , 2014, PloS one.

[4]  Bradley P Carlin,et al.  Guidance on the implementation and reporting of a drug safety Bayesian network meta‐analysis , 2014, Pharmaceutical statistics.

[5]  Sylwia Bujkiewicz,et al.  Presentational approaches used in the UK for reporting evidence synthesis using indirect and mixed treatment comparisons , 2013, Journal of health services research & policy.

[6]  Philippe Ravaud,et al.  Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review , 2013, BMJ.

[7]  J. Cappelleri,et al.  Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review , 2013, BMJ Open.

[8]  Kristian Thorlund,et al.  How to use an article reporting a multiple treatment comparison meta-analysis. , 2012, JAMA.

[9]  Joseph C. Cappelleri,et al.  Use of Mixed Treatment Comparisons in Systematic Reviews , 2012 .

[10]  Georgia Salanti,et al.  Indirect and mixed‐treatment comparison, network, or multiple‐treatments meta‐analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool , 2012, Research synthesis methods.

[11]  Deborah M Caldwell,et al.  NICE DSU Technical Support Document 7: Evidence Synthesis of Treatment Efficacy in Decision Making: A Reviewer’s Checklist , 2012 .

[12]  Keith Abrams,et al.  Use of Indirect and Mixed Treatment Comparisons for Technology Assessment , 2012, PharmacoEconomics.

[13]  Alex J. Sutton,et al.  Heterogeneity: Subgroups, Meta-Regression, Bias And Bias-Adjustment , 2011 .

[14]  Nicky J Welton,et al.  NICE DSU Technical Support Document 2: A Generalised Linear Modelling Framework for Pairwise and Network Meta-Analysis of Randomised Controlled Trials , 2011 .

[15]  Tianjing Li,et al.  Network meta-analysis-highly attractive but more methodological research is needed , 2011, BMC medicine.

[16]  David C Hoaglin,et al.  Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[17]  Joseph C Cappelleri,et al.  Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[18]  Kristian Thorlund,et al.  Multiple treatment comparison meta-analyses: a step forward into complexity , 2011, Clinical epidemiology.

[19]  J. Hutton,et al.  PHP56 DEMONSTRATING CLINICAL-EFFECTIVENESS USING INDIRECT AND MIXED TREATMENT COMPARISON ANALYSIS: A REVIEW OF MANUFACTURERS' SINGLE TECHNOLOGY APPRAISAL (STA) SUBMISSIONS TO THE NATIONAL INSTITUTE FOR HEALTH AND CLINICAL EXCELLENCE (NICE) , 2011 .

[20]  J. Kusel,et al.  PMS69 THE USE OF MIXED TREATMENT COMPARISONS IN NICE TECHNOLOGY APPRAISALS , 2011 .

[21]  Deborah M Caldwell,et al.  NICE DSU Technical Support Document 4: Inconsistency in Networks of Evidence Based on Randomised Controlled Trials. , 2011 .

[22]  Georgia Salanti,et al.  Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. , 2011, Journal of clinical epidemiology.

[23]  Paula Williamson,et al.  Indirect Comparisons: A Review of Reporting and Methodological Quality , 2010, PloS one.

[24]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[25]  John P.A. Ioannidis,et al.  Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses , 2009, Canadian Medical Association Journal.

[26]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[27]  David Moher,et al.  An evidence-based practice guideline for the peer review of electronic search strategies. , 2009, Journal of clinical epidemiology.

[28]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration , 2009, BMJ : British Medical Journal.

[29]  J. Ioannidis,et al.  The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[30]  A. Misra,et al.  PHP82 USE OF INDIRECT COMPARISON IN HTA SUBMISSIONS , 2009 .

[31]  Douglas G Altman,et al.  Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews , 2009, BMJ : British Medical Journal.

[32]  Georgia Salanti,et al.  Evaluation of networks of randomized trials , 2008, Statistical methods in medical research.

[33]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[34]  A. Dhar,et al.  National Institute for Health and Clinical Excellence , 2005 .

[35]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[36]  T. Lumley.  Network meta‐analysis for indirect treatment comparisons , 2002, Statistics in medicine.

[37]  S D Walter,et al.  The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. , 1997, Journal of clinical epidemiology.

[38]  A Whitehead,et al.  Borrowing strength from external trials in a meta-analysis. , 1996, Statistics in medicine.

[39]  Indirect comparisons Methods and validity , 2022 .