Evidence Synthesis for Decision Making 4

Inconsistency can be thought of as a conflict between “direct” evidence on a comparison between treatments B and C and “indirect” evidence gained from AC and AB trials. Like heterogeneity, inconsistency is caused by effect modifiers and specifically by an imbalance in the distribution of effect modifiers in the direct and indirect evidence. Defining inconsistency as a property of loops of evidence, the relation between inconsistency and heterogeneity and the difficulties created by multiarm trials are described. We set out an approach to assessing consistency in 3-treatment triangular networks and in larger circuit structures, its extension to certain special structures in which independent tests for inconsistencies can be created, and describe methods suitable for more complex networks. Sample WinBUGS code is given in an appendix. Steps that can be taken to minimize the risk of drawing incorrect conclusions from indirect comparisons and network meta-analysis are the same steps that will minimize heterogeneity in pairwise meta-analysis. Empirical indicators that can provide reassurance and the question of how to respond to inconsistency are also discussed.

[1]  G. Lu,et al.  Assessing Evidence Inconsistency in Mixed Treatment Comparisons , 2006 .

[2]  S Dias,et al.  Checking consistency in mixed treatment comparison meta‐analysis , 2010, Statistics in medicine.

[3]  A. Cipriani,et al.  Validity of indirect comparisons in meta-analysis , 2007, The Lancet.

[4]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[5]  Deborah M Caldwell,et al.  Selecting the best scale for measuring treatment effect in a network meta‐analysis: a case study in childhood nocturnal enuresis , 2012, Research synthesis methods.

[6]  Alex J Sutton,et al.  Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study , 2011, BMJ : British Medical Journal.

[7]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[8]  David J. Spiegelhalter,et al.  Conflicting evidence in a Bayesian synthesis of surveillance data to estimate human immunodeficiency virus prevalence , 2008 .

[9]  Alex J. Sutton,et al.  Heterogeneity: Subgroups, Meta-Regression, Bias And Bias-Adjustment , 2011 .

[10]  V. Hasselblad,et al.  Meta-analysis of Multitreatment Studies , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[11]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[12]  Deborah M Caldwell,et al.  NICE DSU Technical Support Document 4: Inconsistency in Networks of Evidence Based on Randomised Controlled Trials. , 2011 .

[13]  Gordon Guyatt,et al.  Meta-analyses of therapies for postmenopausal osteoporosis. IX: Summary of meta-analyses of therapies for postmenopausal osteoporosis. , 2002, Endocrine reviews.

[14]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[15]  Nicky J Welton,et al.  NICE DSU Technical Support Document 2: A Generalised Linear Modelling Framework for Pairwise and Network Meta-Analysis of Randomised Controlled Trials , 2011 .

[16]  Deborah M Caldwell,et al.  Mixed treatment comparison analysis provides internally coherent treatment effect estimates based on overviews of reviews and can reveal inconsistency. , 2010, Journal of clinical epidemiology.

[17]  Georgia Salanti,et al.  Evaluation of networks of randomized trials , 2008, Statistical methods in medical research.

[18]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 5: The Baseline Natural History Model. (5), , 2013 .

[19]  D G Altman,et al.  Indirect comparisons of competing interventions. , 2005, Health technology assessment.

[20]  Douglas G Altman,et al.  Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews , 2009, BMJ : British Medical Journal.

[21]  Nicky J Welton,et al.  Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta‐analysis , 2010 .

[22]  J. Geddes,et al.  Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis , 2009, The Lancet.

[23]  S D Walter,et al.  The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. , 1997, Journal of clinical epidemiology.

[24]  David J Spiegelhalter,et al.  Bias modelling in evidence synthesis , 2009, Journal of the Royal Statistical Society. Series A,.

[25]  T. Lumley.  Network meta‐analysis for indirect treatment comparisons , 2002, Statistics in medicine.

[26]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[27]  A Haycox,et al.  Early thrombolysis for the treatment of acute myocardial infarction: a systematic review and economic evaluation. , 2003, Health technology assessment.

[28]  Jonathan J Deeks,et al.  Issues in the selection of a summary statistic for meta‐analysis of clinical trials with binary outcomes , 2002, Statistics in medicine.

[29]  A Whitehead,et al.  Borrowing strength from external trials in a meta-analysis. , 1996, Statistics in medicine.

[30]  Guobing Lu,et al.  Modeling between-trial variance structure in mixed treatment comparisons. , 2009, Biostatistics.

[31]  Ingram Olkin,et al.  Stochastically dependent effect sizes. , 1994 .

[32]  David M. Eddy,et al.  Meta-analysis by the confidence profile method , 1992 .

[33]  Alex J. Sutton,et al.  Heterogeneity: Subgroups, Meta-Regression, Bias And Bias-Adjustment , 2011 .

[34]  J. Boura,et al.  Primary angioplasty versus intravenous thrombolytic therapy for acute myocardial infarction : a quantitative review of 23 randomised trials , 2022 .

[35]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 2 , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[36]  Catherine H Mercer,et al.  Estimates of human immunodeficiency virus prevalence and proportion diagnosed based on Bayesian multiparameter synthesis of surveillance data , 2008 .

[37]  Douglas G Altman,et al.  Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses , 2003, BMJ : British Medical Journal.