Evidence Synthesis for Decision Making 7

This checklist is for the review of evidence syntheses for treatment efficacy used in decision making based on either efficacy or cost-effectiveness. It is intended to be used for pairwise meta-analysis, indirect comparisons, and network meta-analysis, without distinction. It does not generate a quality rating and is not prescriptive. Instead, it focuses on a series of questions aimed at revealing the assumptions that the authors of the synthesis are expecting readers to accept, the adequacy of the arguments authors advance in support of their position, and the need for further analyses or sensitivity analyses. The checklist is intended primarily for those who review evidence syntheses, including indirect comparisons and network meta-analyses, in the context of decision making but will also be of value to those submitting syntheses for review, whether to decision-making bodies or journals. The checklist has 4 main headings: A) definition of the decision problem, B) methods of analysis and presentation of results, C) issues specific to network synthesis, and D) embedding the synthesis in a probabilistic cost-effectiveness model. The headings and implicit advice follow directly from the other tutorials in this series. A simple table is provided that could serve as a pro forma checklist.

[1]  Nicky J Welton,et al.  How valuable are multiple treatment comparison methods in evidence-based health-care evaluation? , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[2]  Nicola J Cooper,et al.  Mixed comparison of stroke prevention treatments in individuals with nonrheumatic atrial fibrillation. , 2006, Archives of internal medicine.

[3]  Alex J. Sutton,et al.  Multiparameter evidence synthesis in epidemiology and medical decision‐making: current approaches , 2006 .

[4]  Douglas G Altman,et al.  Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study , 2008, BMJ : British Medical Journal.

[5]  D. Altman,et al.  Assessing Risk of Bias in Included Studies , 2008 .

[6]  Douglas G Altman,et al.  Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses , 2003, BMJ : British Medical Journal.

[7]  Alex J. Sutton,et al.  Heterogeneity: Subgroups, Meta-Regression, Bias And Bias-Adjustment , 2011 .

[8]  Gordon H Guyatt,et al.  Incorporating multiple interventions in meta-analysis: an evaluation of the mixed treatment comparison with the adjusted indirect comparison , 2009, Trials.

[9]  A E Ades,et al.  Meta‐analysis of mixed treatment comparisons at multiple follow‐up times , 2007, Statistics in medicine.

[10]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[11]  Nicky J Welton,et al.  Mixed treatment comparison meta-analysis of complex interventions: psychological interventions in coronary heart disease. , 2009, American journal of epidemiology.

[12]  Paula Williamson,et al.  Indirect Comparisons: A Review of Reporting and Methodological Quality , 2010, PloS one.

[13]  David R. Jones,et al.  Bayesian methods in health technology assessment: a review. , 2000, Health technology assessment.

[14]  Georgia Salanti,et al.  Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. , 2011, Journal of clinical epidemiology.

[15]  Joseph C Cappelleri,et al.  Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[16]  K R Abrams,et al.  Modelling publication bias in meta-analysis: a review. , 2000, Statistical methods in medical research.

[17]  Alex J Sutton,et al.  Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study , 2011, BMJ : British Medical Journal.

[18]  Nicky J Welton,et al.  Study designs to detect sponsorship and other biases in systematic reviews. , 2010, Journal of clinical epidemiology.

[19]  G H Guyatt,et al.  Guidelines for reading literature reviews. , 1988, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[20]  Alexander J Sutton,et al.  What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. , 2004, Statistics in medicine.

[21]  D. Cook,et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? , 1998, The Lancet.

[22]  Douglas G. Altman,et al.  Models for potentially biased evidence in meta‐analysis using empirically based priors , 2009 .

[23]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 5: The Baseline Natural History Model. (5), , 2013 .

[24]  G. Guyatt,et al.  Grading quality of evidence and strength of recommendations , 2004, BMJ : British Medical Journal.

[25]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[26]  David C Hoaglin,et al.  Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[27]  G. Guyatt,et al.  GRADE: an emerging consensus on rating quality of evidence and strength of recommendations , 2008, BMJ : British Medical Journal.

[28]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 1: Introduction. Medical Decision Making, , 2013 .

[29]  Deborah M Caldwell,et al.  Validity of indirect comparisons in meta-analysis , 2007, The Lancet.

[30]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[31]  Byron Jones,et al.  Statistical approaches for conducting network meta‐analysis in drug development , 2011, Pharmaceutical statistics.

[32]  Nicky J Welton,et al.  Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta‐analysis , 2010 .

[33]  J. Geddes,et al.  Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis , 2009, The Lancet.

[34]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[35]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[36]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[37]  A E Ades,et al.  Accounting for correlation in network meta‐analysis with multi‐arm trials , 2012, Research synthesis methods.

[38]  A E Ades,et al.  Synthesis of survival and disease progression outcomes for health technology assessment of cancer therapies , 2010, Research Synthesis Methods.

[39]  A E Ades,et al.  Mixed treatment comparison with multiple outcomes reported inconsistently across trials: Evaluation of antivirals for treatment of influenza A and B , 2008, Statistics in medicine.

[40]  Roger Chou,et al.  Initial highly-active antiretroviral therapy with a protease inhibitor versus a non-nucleoside reverse transcriptase inhibitor: discrepancies between direct and indirect meta-analyses , 2006, The Lancet.

[41]  D. Spiegelhalter,et al.  Bayesian Approaches to Clinical Trials and Health-Care Evaluation: Spiegelhalter/Clinical Trials and Health-Care Evaluation , 2004 .

[42]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 6: Embedding Evidence Synthesis in Probabilistic Cost-effectiveness Analysis. , 2013 .

[43]  Julia Earnshaw,et al.  NICE Guide to the Methods of Technology Appraisal , 2012, PharmacoEconomics.

[44]  P. Gøtzsche,et al.  Why we need a broad perspective on meta-analysis , 2000, BMJ : British Medical Journal.

[45]  Peter Jüni,et al.  Direction and impact of language bias in meta-analyses of controlled trials: empirical study. , 2002, International journal of epidemiology.

[46]  Christian Gluud,et al.  Reported Methodologic Quality and Discrepancies between Large and Small Randomized Trials in Meta-Analyses , 2001, Annals of Internal Medicine.

[47]  D. Sackett,et al.  Cochrane Collaboration , 1994, BMJ.

[48]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making Inconsistency in Networks of Evidence Based on Randomized Controlled Trials. , 2013 .

[49]  Steven N Goodman,et al.  Systematic reviews are not biased by results from trials stopped early for benefit. , 2008, Journal of clinical epidemiology.

[50]  Jonathan J Deeks,et al.  Much ado about nothing: a comparison of the performance of meta‐analytical methods with rare events , 2007, Statistics in medicine.

[51]  Ian R. White,et al.  Multivariate Random-effects Meta-regression: Updates to Mvmeta , 2011 .

[52]  Neil Hawkins,et al.  How Far Do You Go? Efficient Searching for Indirect Evidence , 2009, Medical decision making : an international journal of the Society for Medical Decision Making.

[53]  D. Collet Modelling Survival Data in Medical Research , 2004 .

[54]  David J Spiegelhalter,et al.  Bias modelling in evidence synthesis , 2009, Journal of the Royal Statistical Society. Series A,.

[55]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[56]  Boris Freidlin,et al.  Bias and trials stopped early for benefit. , 2010, JAMA.

[57]  David Moher,et al.  Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews , 2007, BMC medical research methodology.

[58]  David Collett Modelling Survival Data in Medical Research , 1994 .

[59]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 2 , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[60]  M. A. Best Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[61]  Igor Sartori,et al.  Empirical Evidence of Bias. Dimensions of Methodological Quality Associated with Estimates of Treatment Effects in Controlled Trials , 1996 .

[62]  Michele Tarsilla Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[63]  Nicky J Welton,et al.  NICE DSU Technical Support Document 1: Introduction to Evidence Synthesis for Decision Making , 2011 .

[64]  Douglas G Altman,et al.  The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews , 2010, BMJ : British Medical Journal.

[65]  S. Greenland,et al.  Methods for trend estimation from summarized dose-response data, with applications to meta-analysis. , 1992, American journal of epidemiology.

[66]  Bradley P Carlin,et al.  Bias and trials stopped early for benefit. , 2010, JAMA.

[67]  J. Higgins Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration , 2011 .

[68]  Douglas G Altman,et al.  Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews , 2009, BMJ : British Medical Journal.

[69]  Nicky J Welton,et al.  Network meta-analysis with competing risk outcomes. , 2010, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.