Additional considerations are required when preparing a protocol for a systematic review with multiple interventions.

OBJECTIVES The number of systematic reviews that aim to compare multiple interventions using network meta-analysis is increasing. In this study, we highlight aspects of a standard systematic review protocol that may need modification when multiple interventions are to be compared. STUDY DESIGN AND SETTING We take the protocol format suggested by Cochrane for a standard systematic review as our reference and compare the considerations for a pairwise review with those required for a valid comparison of multiple interventions. We suggest new sections for protocols of systematic reviews including network meta-analyses with a focus on how to evaluate their assumptions. We provide example text from published protocols to exemplify the considerations. CONCLUSION Standard systematic review protocols for pairwise meta-analyses need extensions to accommodate the increased complexity of network meta-analysis. Our suggested modifications are widely applicable to both Cochrane and non-Cochrane systematic reviews involving network meta-analyses.

[1]  Dan Jackson,et al.  A design-by-treatment interaction model for network meta-analysis with random inconsistency effects , 2014, Statistics in medicine.

[2]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[3]  J. Higgins,et al.  Guide to the Contents of a Cochrane Protocol and Review , 2008 .

[4]  Panagiota Spyridonos,et al.  Graphical Tools for Network Meta-Analysis in STATA , 2013, PloS one.

[5]  L. Trinquart,et al.  Meta-Analysis of a Complex Network of Non-Pharmacological Interventions: The Example of Femoral Neck Fracture , 2016, PLoS ONE.

[6]  Protocol template for a Cochrane intervention review that compares multiple interventions , 2014 .

[7]  J. Geddes,et al.  Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis , 2009, The Lancet.

[8]  Joseph C Cappelleri,et al.  Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. , 2014, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[9]  Douglas G. Altman,et al.  Analysing Data and Undertaking Meta‐Analyses , 2019 .

[10]  R. Ehrenkranz.  In utero antiepileptic drug exposure: Fetal death and malformations , 2007 .

[11]  Andrea Cipriani,et al.  Conceptual and Technical Challenges in Network Meta-analysis , 2013, Annals of Internal Medicine.

[12]  Catrin Tudur Smith,et al.  Assessing key assumptions of network meta‐analysis: a review of methods , 2013, Research synthesis methods.

[13]  Neil Hawkins,et al.  How Far Do You Go? Efficient Searching for Indirect Evidence , 2009, Medical decision making : an international journal of the Society for Medical Decision Making.

[14]  Anna Chaimani,et al.  Using network meta‐analysis to evaluate the existence of small‐study effects in a network of interventions , 2012, Research synthesis methods.

[15]  C. Bearer.  Malformation risks of antiepileptic drugs in pregnancy: a prospective study from the UK Epilepsy and Pregnancy Register , 2007 .

[16]  G. Lu,et al.  Assessing Evidence Inconsistency in Mixed Treatment Comparisons , 2006 .

[17]  J. Liporace.  Management issues for women with epilepsy. , 1999, Neurology.

[18]  Tianjing Li,et al.  Outcomes in Cochrane Systematic Reviews Addressing Four Common Eye Conditions: An Evaluation of Completeness and Comparability , 2014, PloS one.

[19]  AE Ades,et al.  Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies‡ , 2012, Research synthesis methods.

[20]  L. Holmes,et al.  Increased frequency of isolated cleft palate in infants exposed to lamotrigine during pregnancy , 2008, Neurology.

[21]  Dan Jackson,et al.  Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression‡ , 2012, Research synthesis methods.

[22]  Russell L. Kolts,et al.  Research design features and patient characteristics associated with the outcome of antidepressant clinical trials. , 2004, The American journal of psychiatry.

[23]  Alex J. Sutton,et al.  Heterogeneity: Subgroups, Meta-Regression, Bias And Bias-Adjustment , 2011 .

[24]  T. Furukawa,et al.  Antidepressants, benzodiazepines and azapirones for panic disorder in adults: a network meta-analysis , 2017 .

[25]  Kristian Thorlund,et al.  The Quality of Reporting Methods and Results in Network Meta-Analyses: An Overview of Reviews and Suggestions for Improvement , 2014, PloS one.

[26]  R. DeRubeis,et al.  Antidepressant drug effects and depression severity: a patient-level meta-analysis. , 2010, JAMA.

[27]  Mohammad Hassan Murad,et al.  A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis , 2015, BMJ : British Medical Journal.

[28]  Philippe Ravaud,et al.  Uncertainty in Treatment Rankings: Reanalysis of Network Meta-analyses of Randomized Trials , 2016, Annals of Internal Medicine.

[29]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 2 , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[30]  Alex J Sutton,et al.  Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. , 2008, Journal of clinical epidemiology.

[31]  Michele Tarsilla.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[32]  Gerta Rücker,et al.  Ranking treatments in frequentist network meta-analysis works without resampling methods , 2015, BMC Medical Research Methodology.

[33]  Nicky J Welton,et al.  Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study. , 2013, International journal of epidemiology.

[34]  Alex J Sutton,et al.  Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study , 2011, BMJ : British Medical Journal.

[35]  Huseyin Naci,et al.  Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers , 2013, BMC Medicine.

[36]  Tianjing Li,et al.  Network meta-analysis-highly attractive but more methodological research is needed , 2011, BMC medicine.

[37]  Georgia Salanti,et al.  Evaluation of networks of randomized trials , 2008, Statistical methods in medical research.

[38]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[39]  D. Moher,et al.  Comparative safety of anti-epileptic drugs among infants and children exposed in utero or during breastfeeding: protocol for a systematic review and network meta-analysis , 2014, BMC Medicine.

[40]  V. Cornelius,et al.  Comparative efficacy and acceptability of drug treatments for bipolar depression: a multiple‐treatments meta‐analysis , 2014, Acta psychiatrica Scandinavica.

[41]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 7 , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[42]  Dimitris Mavridis,et al.  A fully Bayesian application of the Copas selection model for publication bias extended to network meta‐analysis , 2013, Statistics in medicine.

[43]  M. Eadie,et al.  Foetal malformations and seizure control: 52 months data of the Australian Pregnancy Registry , 2006, European journal of neurology.

[44]  A E Ades,et al.  Accounting for correlation in network meta‐analysis with multi‐arm trials , 2012, Research synthesis methods.

[45]  Gerald Gartlehner,et al.  [GRADE guidelines: 1. Introduction - GRADE evidence profiles and summary of findings tables]. , 2012, Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen.

[46]  Ronald C Kessler,et al.  The economic burden of depression in the United States: how did it change between 1990 and 2000? , 2003, The Journal of clinical psychiatry.

[47]  Deborah M Caldwell,et al.  Novel presentational approaches were developed for reporting network meta-analysis. , 2014, Journal of clinical epidemiology.

[48]  Julian P T Higgins,et al.  Evaluation of inconsistency in networks of interventions. , 2013, International journal of epidemiology.

[49]  J. Copas.  What works?: selectivity models and meta‐analysis , 1999 .

[50]  Georgia Salanti,et al.  Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. , 2011, Journal of clinical epidemiology.

[51]  Joseph Beyene,et al.  Bias in identification of the best treatment in a Bayesian network meta-analysis for binary outcome: a simulation study , 2014, Clinical epidemiology.

[52]  Nicky J Welton,et al.  A selection model for accounting for publication bias in a full network meta‐analysis , 2014, Statistics in medicine.

[53]  K. Meador,et al.  Antiepileptic drug use in women of childbearing age , 2009, Epilepsy & Behavior.

[54]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[55]  D. Chadwick,et al.  Additional educational needs in children born to mothers with epilepsy , 2001, Journal of neurology, neurosurgery, and psychiatry.

[56]  A. Kiss,et al.  Does inclusion of a placebo arm influence response to active antidepressant treatment in randomized controlled trials? Results from pooled and meta-analyses. , 2010, The Journal of clinical psychiatry.

[57]  Maurizio Fava,et al.  Does the probability of receiving placebo influence clinical trial outcome? A meta-regression of double-blind, randomized clinical trials in MDD , 2009, European Neuropsychopharmacology.

[58]  Dimitris Mavridis,et al.  Allowing for uncertainty due to missing continuous outcome data in pairwise and network meta‐analysis , 2015, Statistics in medicine.

[59]  S Dias,et al.  Checking consistency in mixed treatment comparison meta‐analysis , 2010, Statistics in medicine.

[60]  N. Welton,et al.  Addressing between‐study heterogeneity and inconsistency in mixed treatment comparisons: Application to stroke prevention treatments in individuals with non‐rheumatic atrial fibrillation , 2009, Statistics in medicine.

[61]  S. Roose,et al.  Does Study Design Influence Outcome? , 2009, Psychotherapy and Psychosomatics.

[62]  V. Hiilesmaa,et al.  Normal intelligence in children with prenatal exposure to carbamazepine , 2004, Neurology.

[63]  Douglas G. Altman,et al.  Analysing data and undertaking meta‐analyses , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[64]  J P Vandenbroucke,et al.  Bias in meta-analysis detected by a simple, graphical test , 1998 .

[65]  Andrew W Lee,et al.  Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. , 2014, Journal of clinical epidemiology.

[66]  Alex J. Sutton,et al.  Evidence Synthesis for Decision Making 4 , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[67]  Vikram Patel,et al.  Efficacy of antidepressants and benzodiazepines in minor depression: systematic review and meta-analysis , 2011, British Journal of Psychiatry.

[68]  D. Moher,et al.  The PRISMA Extension Statement , 2015, Annals of Internal Medicine.

[69]  J. Stockman.  Cognitive Function at 3 Years of Age after Fetal Exposure to Antiepileptic Drugs , 2011 .

[70]  Dimitris Mavridis,et al.  Network meta‐analysis models to account for variability in treatment definitions: application to dose effects , 2013, Statistics in medicine.

[71]  Kristian Thorlund,et al.  The effects of excluding treatments from network meta-analyses: survey , 2013, BMJ : British Medical Journal.

[72]  Christopher H. Schmid,et al.  Characteristics of Networks of Interventions: A Description of a Database of 186 Published Networks , 2014, PloS one.

[73]  Nicky J Welton,et al.  Synthesis of evidence on heterogeneous interventions with multiple outcomes recorded over multiple follow‐up times reported inconsistently: a smoking cessation case‐study , 2014 .

[74]  D. Caldwell.  An overview of conducting systematic reviews with network meta-analysis , 2014, Systematic Reviews.

[75]  S D Walter,et al.  The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. , 1997, Journal of clinical epidemiology.

[76]  J. Clayton-Smith,et al.  A note on Pierre Marie (1853–1940) , 2004, Journal of Neurology, Neurosurgery & Psychiatry.

[77]  Anna Chaimani,et al.  Evaluating the Quality of Evidence from a Network Meta-Analysis , 2014, PloS one.

[78]  Julian Pt Higgins,et al.  Evaluating the impact of imputations for missing participant outcome data in a network meta-analysis , 2013, Clinical trials.

[79]  D. Moher,et al.  PROSPERO at one year: an evaluation of its utility , 2013, Systematic Reviews.

[80]  Georgia Salanti,et al.  Indirect and mixed‐treatment comparison, network, or multiple‐treatments meta‐analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool , 2012, Research synthesis methods.

[81]  Joseph C Cappelleri,et al.  Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 1. , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[82]  J. Mann,et al.  Benefits from antidepressants: synthesis of 6-week patient-level outcomes from double-blind placebo-controlled randomized trials of fluoxetine and venlafaxine. , 2012, Archives of general psychiatry.

[83]  Nicky J Welton,et al.  How valuable are multiple treatment comparison methods in evidence-based health-care evaluation? , 2011, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[84]  Sofia Dias,et al.  Extending Treatment Networks in Health Technology Assessment: How Far Should We Go? , 2015, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[85]  T. Furukawa,et al.  Comparative efficacy and acceptability of first-generation and second-generation antidepressants in the acute treatment of major depression: protocol for a network meta-analysis , 2016, BMJ Open.

[86]  Julian P T Higgins,et al.  A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. , 2009, Journal of clinical epidemiology.

[87]  K. Meador,et al.  In utero antiepileptic drug exposure , 2006, Neurology.

[88]  Joseph Lau,et al.  Innovations in Data Collection, Management, and Archiving for Systematic Reviews , 2015, Annals of Internal Medicine.

[89]  D. Rubin,et al.  In utero exposure to phenobarbital and intelligence deficits in adult men. , 1995, JAMA.