Evaluation of networks of randomized trials

Randomized trials may be designed and interpreted as single experiments or they may be seen in the context of other similar or relevant evidence. The amount and complexity of available randomized evidence vary for different topics. Systematic reviews may be useful in identifying gaps in the existing randomized evidence, pointing to discrepancies between trials, and planning future trials. A new, promising, but also very much debated extension of systematic reviews, mixed treatment comparison (MTC) meta-analysis, has become increasingly popular recently. MTC meta-analysis may have value in interpreting the available randomized evidence from networks of trials and can rank many different treatments, going beyond focusing on simple pairwise-comparisons. Nevertheless, the evaluation of networks also presents special challenges and caveats. In this article, we review the statistical methodology for MTC meta-analysis. We discuss the concept of inconsistency and methods that have been proposed to evaluate it as well as the methodological gaps that remain. We introduce the concepts of network geometry and asymmetry, and propose metrics for the evaluation of the asymmetry. Finally, we discuss the implications of inconsistency, network geometry and asymmetry in informing the planning of future trials.

[1]  Douglas G Altman,et al.  Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses , 2003, BMJ : British Medical Journal.

[2]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[3]  A. Magurran.  Ecological Diversity and Its Measurement , 1988, Springer Netherlands.

[4]  Michael S Saag,et al.  Treatment for adult HIV infection: 2006 recommendations of the International AIDS Society--USA panel. , 2006, Topics in HIV medicine : a publication of the International AIDS Society, USA.

[5]  J. T. Thigpen.  Survival Benefits With Diverse Chemotherapy Regimens for Ovarian Cancer: Meta-analysis of Multiple Treatments , 2008 .

[6]  G. Guyatt,et al.  Users' Guides to the Medical Literature: IX. A Method for Grading Health Care Recommendations , 1995 .

[7]  Douglas G. Altman,et al.  Systematic Reviews in Health Care , 2001 .

[8]  J. Jansen.  Self-monitoring of glucose in type 2 diabetes mellitus:a Bayesian meta-analysis of direct and indirect comparisons , 2006, Current Medical Research and Opinion.

[9]  Roger Chou,et al.  Initial highly-active antiretroviral therapy with a protease inhibitor versus a non-nucleoside reverse transcriptase inhibitor: discrepancies between direct and indirect meta-analyses , 2006, The Lancet.

[10]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[11]  Mike Clarke,et al.  Discussion sections in reports of controlled trials published in general medical journals. , 2002, JAMA.

[12]  Douglas G Altman,et al.  Systematic reviews in health care: Assessing the quality of controlled clinical trials. , 2001, BMJ.

[13]  R J Cook,et al.  Users' guides to the medical literature. IX. A method for grading health care recommendations. Evidence-Based Medicine Working Group. , 1995, JAMA.

[14]  B. Djulbegovic,et al.  The quality of medical evidence in hematology-oncology. , 1999, The American journal of medicine.

[15]  T. Lumley.  Network meta‐analysis for indirect treatment comparisons , 2002, Statistics in medicine.

[16]  J. Ioannidis,et al.  Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[17]  J. Boura,et al.  Primary angioplasty versus intravenous thrombolytic therapy for acute myocardial infarction : a quantitative review of 23 randomised trials , 2022 .

[18]  G. Lu,et al.  Assessing Evidence Inconsistency in Mixed Treatment Comparisons , 2006 .

[19]  M. Abrahamowicz,et al.  Mortality Rates in Elderly Patients Who Take Different Angiotensin-Converting Enzyme Inhibitors after Acute Myocardial Infarction: A Class Effect? , 2004, Annals of Internal Medicine.

[20]  C. Furberg,et al.  Are all angiotensin-converting enzyme inhibitors interchangeable? , 2001, Journal of the American College of Cardiology.

[21]  R. Matthews,et al.  What are the implications of optimism bias in clinical research? , 2006, The Lancet.

[22]  Alan Brennan,et al.  Using mixed treatment comparisons and meta‐regression to perform indirect comparisons to estimate the efficacy of biologic treatments in rheumatoid arthritis , 2007, Statistics in Medicine.

[23]  Laura Bojke,et al.  Incorporating direct and indirect evidence using bayesian methods: an applied case study in ovarian cancer. , 2006, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[24]  T C Chalmers,et al.  Changes in clinical trials mandated by the advent of meta-analysis. , 1996, Statistics in medicine.

[25]  Nicola J Cooper,et al.  Evidence‐based sample size calculations based upon updated meta‐analysis , 2007, Statistics in medicine.

[26]  Ethan M Balk,et al.  Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. , 2002, JAMA.

[27]  S. Hammer,et al.  A trial comparing nucleoside monotherapy with combination therapy in HIV-infected adults with CD4 cell counts from 200 to 500 per cubic millimeter. AIDS Clinical Trials Group Study 175 Study Team. , 1996, The New England journal of medicine.

[28]  T C Chalmers,et al.  Cumulative meta-analysis of therapeutic trials for myocardial infarction. , 1992, The New England journal of medicine.

[29]  S. Thompson,et al.  How should meta‐regression analyses be undertaken and interpreted? , 2002, Statistics in medicine.

[30]  D G Altman,et al.  Indirect comparisons of competing interventions. , 2005, Health technology assessment.

[31]  Stefan Leucht,et al.  Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. , 2006, The American journal of psychiatry.

[32]  F Mosteller,et al.  Meta-analysis of multiple outcomes by regression with random effects. , 1998, Statistics in medicine.

[33]  M A Fischl,et al.  A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less. AIDS Clinical Trials Group 320 Study Team. , 1997, The New England journal of medicine.

[34]  D M Eddy,et al.  Synthesis of environmental evidence: nitrogen dioxide epidemiology studies. , 1992, Journal of the Air & Waste Management Association.

[35]  P. Raina,et al.  Challenges in Systematic Reviews That Evaluate Drug Efficacy or Effectiveness , 2005, Annals of Internal Medicine.

[36]  Alexander Tsertsvadze,et al.  Systematic reviews: when is an update an update? , 2006, The Lancet.

[37]  Alex J. Sutton,et al.  Multiparameter evidence synthesis in epidemiology and medical decision‐making: current approaches , 2006 .

[38]  Douglas G Altman,et al.  Epidemiology and reporting of randomised trials published in PubMed journals , 2005, The Lancet.

[39]  I Chalmers,et al.  Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? , 1998, JAMA.

[40]  J. Ioannidis,et al.  Randomised trials comparing chemotherapy regimens for advanced non-small cell lung cancer: biases and evolution over time. , 2003, European journal of cancer.

[41]  A Whitehead,et al.  Borrowing strength from external trials in a meta-analysis. , 1996, Statistics in medicine.

[42]  Stuart G Baker,et al.  The transitive fallacy for randomized trials: If A bests B and B bests C in separate trials, is A better than C? , 2002, BMC medical research methodology.

[43]  B. Djulbegovic,et al.  Pharmaceutical industry sponsorship and research outcome and quality: systematic review , 2003, BMJ : British Medical Journal.

[44]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[45]  R. Lilford.  Ethics of clinical trials from a bayesian and decision analytic perspective: whose equipoise is it anyway? , 2003, BMJ : British Medical Journal.

[46]  Thomas Lumley,et al.  Health outcomes associated with various antihypertensive therapies used as first-line agents: a network meta-analysis. , 2003, JAMA.

[47]  Tania B. Huedo-Medina,et al.  Assessing heterogeneity in meta-analysis: Q statistic or I2 index? , 2006, Psychological methods.

[48]  Nicola J Cooper,et al.  Mixed comparison of stroke prevention treatments in individuals with nonrheumatic atrial fibrillation. , 2006, Archives of internal medicine.

[49]  J. Ioannidis,et al.  Indirect comparisons: the mesh and mess of clinical trials , 2006, The Lancet.

[50]  A. Webster,et al.  Systematic review and meta-analysis of antifungal agents for preventing fungal infections in liver transplant recipients , 2006, European Journal of Clinical Microbiology and Infectious Diseases.

[51]  D. Altman,et al.  Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice , 2002, Journal of health services research & policy.

[52]  L. Becker,et al.  The Cochrane Library and bronchiolitis: an umbrella review , 2006 .

[53]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[54]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[55]  J. Ioannidis,et al.  Clinical trials in sub-Saharan Africa and established standards of care: a systematic review of HIV, tuberculosis, and malaria trials. , 2004, JAMA.

[56]  D. Beek,et al.  A systematic review of Helicobacter pylori eradication therapy—the impact of antimicrobial resistance on eradication rates , 1999, Alimentary pharmacology & therapeutics.

[57]  H S Sacks,et al.  The relationship between study design, results, and reporting of randomized clinical trials of HIV infection. , 1997, Controlled clinical trials.

[58]  D L Sackett,et al.  Users' Guides to the Medical Literature: XIX. Applying clinical trial results B. Guidelines for determining whether a drug is exerting (more than) a class effect. , 1999, JAMA.

[59]  S. Hurlbert.  The Nonconcept of Species Diversity: A Critique and Alternative Parameters. , 1971, Ecology.

[60]  L. Stone,et al.  The checkerboard score and species distributions , 1990, Oecologia.

[61]  V. Berger.  Is the Jadad score the proper evaluation of trials? , 2006, The Journal of rheumatology.

[62]  David M. Eddy,et al.  Meta-analysis by the confidence profile method , 1992 .

[63]  B. Psaty,et al.  Principles for evidence-based drug formulary policy , 2005, Journal of General Internal Medicine.

[64]  J. Gray.  Evidence-Based Healthcare , 1997 .

[65]  M. Parmar,et al.  Meta-analysis of the literature or of individual patient data: is there a difference? , 1993, The Lancet.

[66]  M. Pittler.  Systematic Reviews in Health Care: Meta‐analysis in Context , 2010 .

[67]  L. Gluud.  Bias in clinical intervention research. , 2006, American journal of epidemiology.

[68]  Christopher H Schmid,et al.  Summing up evidence: one answer is not always enough , 1998, The Lancet.

[69]  P. Austin,et al.  Effectiveness of statins for secondary prevention in elderly patients after acute myocardial infarction: an evaluation of class effect , 2005, Canadian Medical Association Journal.