A scoping review of indirect comparison methods and applications using individual patient data

BackgroundSeveral indirect comparison methods, including network meta-analyses (NMAs), using individual patient data (IPD) have been developed to synthesize evidence from a network of trials. Although IPD indirect comparisons are published with increasing frequency in health care literature, there is no guidance on selecting the appropriate methodology and on reporting the methods and results.MethodsIn this paper we examine the methods and reporting of indirect comparison methods using IPD. We searched MEDLINE, Embase, the Cochrane Library, and CINAHL from inception until October 2014. We included published and unpublished studies reporting a method, application, or review of indirect comparisons using IPD and at least three interventions.ResultsWe identified 37 papers, including a total of 33 empirical networks. Of these, only 9 (27 %) IPD-NMAs reported the existence of a study protocol, whereas 3 (9 %) studies mentioned that protocols existed without providing a reference. The 33 empirical networks included 24 (73 %) IPD-NMAs and 9 (27 %) matching adjusted indirect comparisons (MAICs). Of the 21 (64 %) networks with at least one closed loop, 19 (90 %) were IPD-NMAs, 13 (68 %) of which evaluated the prerequisite consistency assumption, and only 5 (38 %) of the 13 IPD-NMAs used statistical approaches. The median number of trials included per network was 10 (IQR 4–19) (IPD-NMA: 15 [IQR 8–20]; MAIC: 2 [IQR 3–5]), and the median number of IPD trials included in a network was 3 (IQR 1–9) (IPD-NMA: 6 [IQR 2–11]; MAIC: 2 [IQR 1–2]). Half of the networks (17; 52 %) applied Bayesian hierarchical models (14 one-stage, 1 two-stage, 1 used IPD as an informative prior, 1 unclear-stage), including either IPD alone or with aggregated data (AD). Models for dichotomous and continuous outcomes were available (IPD alone or combined with AD), as were models for time-to-event data (IPD combined with AD).ConclusionsOne in three indirect comparison methods modeling IPD adjusted results from different trials to estimate effects as if they had come from the same, randomized, population. Key methodological and reporting elements (e.g., evaluation of consistency, existence of study protocol) were often missing from an indirect comparison paper.

[1]  Mei Lu,et al.  Comparative Efficacy of Guanfacine Extended Release Versus Atomoxetine for the Treatment of Attention-Deficit/Hyperactivity Disorder in Children and Adolescents: Applying Matching-Adjusted Indirect Comparison Methodology , 2013, CNS Drugs.

[2]  A. Sutton,et al.  Mixed treatment comparisons using aggregate and individual participant level data , 2012, Statistics in medicine.

[3]  J. Jansen,et al.  Efficacy of once-daily indacaterol relative to alternative bronchodilators in COPD: a patient-level mixed treatment comparison. , 2012, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[4]  A Whitehead,et al.  Meta‐analysis of continuous outcome data from individual patients , 2001, Statistics in medicine.

[5]  Jeroen P. Jansen,et al.  PRM2 Network Meta-Analysis of Individual and Aggregate Level Data , 2012 .

[6]  David W. Hosmer,et al.  Applied Logistic Regression , 1991 .

[7]  Stephen W. Raudenbush,et al.  Analyzing effect sizes: Random-effects models. , 2009 .

[8]  Mark C Simmonds,et al.  Meta-analysis of individual patient data from randomized trials: a review of methods used in practice , 2005, Clinical trials.

[9]  Bradley P Carlin,et al.  Incorporation of individual‐patient data in network meta‐analysis for multiple continuous endpoints, with application to diabetes treatment , 2015, Statistics in medicine.

[10]  Richard M Nixon,et al.  Network meta-analysis combining individual patient and aggregate data from a mixture of study designs with an application to pulmonary arterial hypertension , 2015, BMC Medical Research Methodology.

[11]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[12]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[13]  K. Jack Ishak,et al.  No Head-to-Head Trial? Simulate the Missing Arms , 2012, PharmacoEconomics.

[14]  S Dias,et al.  Checking consistency in mixed treatment comparison meta‐analysis , 2010, Statistics in medicine.

[15]  Andrew W Lee,et al.  Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009. , 2014, Journal of clinical epidemiology.

[16]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[17]  T. Lumley.  Network meta‐analysis for indirect treatment comparisons , 2002, Statistics in medicine.

[18]  Kristian Thorlund,et al.  The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations , 2015, Annals of Internal Medicine.

[19]  Richard D Riley,et al.  Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement , 2015 .

[20]  Eric Q. Wu,et al.  Comparative efficacy of nilotinib and dasatinib in newly diagnosed chronic myeloid leukemia: a matching-adjusted indirect comparison of randomized trials , 2011, Current medical research and opinion.

[21]  J. Reginster,et al.  Bazedoxifene versus oral bisphosphonates for the prevention of nonvertebral fractures in postmenopausal women with osteoporosis at higher risk of fracture: a network meta-analysis. , 2014, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[22]  Beth Devine,et al.  Everolimus and sunitinib for advanced pancreatic neuroendocrine tumors: a matching-adjusted indirect comparison , 2013, Experimental Hematology & Oncology.

[23]  Aaron N. Winn,et al.  An Assessment of the Methodological Quality of Published Network Meta-Analyses: A Systematic Review , 2015, PloS one.

[24]  Harold I Feldman,et al.  Individual patient‐ versus group‐level data meta‐regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head , 2002, Statistics in medicine.

[25]  J. Pignon,et al.  Mixed treatment comparison meta-analysis of altered fractionated radiotherapy and chemotherapy in head and neck cancer. , 2011, Journal of clinical epidemiology.

[26]  Jeroen P Jansen,et al.  Network meta‐analysis of individual and aggregate level data , 2012, Research synthesis methods.

[27]  T. Schnitzer,et al.  Vascular and upper gastrointestinal effects of non-steroidal anti-inflammatory drugs: meta-analyses of individual participant data from randomised trials , 2013, BDJ.

[28]  Paula Williamson,et al.  Indirect Comparisons: A Review of Reporting and Methodological Quality , 2010, PloS one.

[29]  Catrin Tudur Smith,et al.  Assessing the consistency assumption by exploring treatment by covariate interactions in mixed treatment comparison meta‐analysis: individual patient‐level covariates versus aggregate trial‐level covariates , 2012, Statistics in medicine.

[30]  H. Arksey,et al.  Scoping studies: towards a methodological framework , 2005 .

[31]  Gavin Giovannoni,et al.  No Evidence of Disease Activity: Indirect Comparisons of Oral Therapies for the Treatment of Relapsing–Remitting Multiple Sclerosis , 2014, Advances in Therapy.

[32]  Catrin Tudur Smith,et al.  Assessing key assumptions of network meta‐analysis: a review of methods , 2013, Research synthesis methods.

[33]  J. Jansen,et al.  Efficacy of once-daily indacaterol 75 μg relative to alternative bronchodilators in COPD: A study level and a patient level network meta-analysis , 2012, BMC Pulmonary Medicine.

[34]  Areti Angeliki Veroniki,et al.  Methods and characteristics of published network meta-analyses using individual patient data: protocol for a scoping review , 2015, BMJ Open.

[35]  Blair T. Johnson,et al.  Contemporary Approaches to Meta-Analysis in Communication Research , 2008 .

[36]  Catrin Tudur Smith,et al.  Combining individual patient data and aggregate data in mixed treatment comparison meta‐analysis: Individual patient data may be beneficial if only for a subset of trials , 2013, Statistics in medicine.

[37]  J. Pignon,et al.  Meta-analysis of chemotherapy in head and neck cancer (MACH-NC): an update on 93 randomised trials and 17,346 patients. , 2009, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[38]  Christopher H. Schmid,et al.  Characteristics of Networks of Interventions: A Description of a Database of 186 Published Networks , 2014, PloS one.

[39]  Pedro Saramago,et al.  Network meta-analysis of (individual patient) time to event data alongside (aggregate) count data , 2014, BMC Medical Research Methodology.

[40]  D. Parker,et al.  Guidance for conducting systematic scoping reviews , 2015, International journal of evidence-based healthcare.

[41]  A. Marson,et al.  Multiple treatment comparisons in epilepsy monotherapy trials , 2007, Trials.

[42]  L. Mantovani,et al.  The importance of baseline viral load when assessing relative efficacy in treatment-naïve HBeAg-positive chronic hepatitis B: a systematic review and network meta-analysis , 2014, Systematic Reviews.

[43]  H. Akaike.  A new look at the statistical model identification , 1974 .

[44]  S. Bhattacharya,et al.  Second generation endometrial ablation techniques for heavy menstrual bleeding: network meta-analysis , 2012, BMJ : British Medical Journal.

[45]  Joseph C Cappelleri,et al.  Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. , 2014, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[46]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[47]  Julian P T Higgins,et al.  Evaluation of inconsistency in networks of interventions. , 2013, International journal of epidemiology.

[48]  Harris Cooper,et al.  The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. , 2009, Psychological methods.

[49]  N. Ford,et al.  Interventions to promote adherence to antiretroviral therapy in Africa: a network meta-analysis. , 2014, The lancet. HIV.

[50]  François Gueyffier,et al.  Uptake of systematic reviews and meta-analyses based on individual participant data in clinical practice guidelines: descriptive study , 2015, BMJ : British Medical Journal.

[51]  Eric Q. Wu,et al.  Comparative Efficacy of Vildagliptin and Sitagliptin in Japanese Patients with Type 2 Diabetes Mellitus , 2011, Clinical drug investigation.

[52]  J. Cappelleri,et al.  Methods used to conduct and report Bayesian mixed treatment comparisons published in the medical literature: a systematic review , 2013, BMJ Open.

[53]  Georgia Salanti,et al.  Indirect and mixed‐treatment comparison, network, or multiple‐treatments meta‐analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool , 2012, Research synthesis methods.

[54]  Eric Q. Wu,et al.  Comparative Effectiveness Without Head-to-Head Trials , 2012, PharmacoEconomics.

[55]  David Moher,et al.  An evidence-based practice guideline for the peer review of electronic search strategies. , 2009, Journal of clinical epidemiology.

[56]  Irina Proskorovsky,et al.  Simulation and Matching-Based Approaches for Indirect Comparison of Treatments , 2015, PharmacoEconomics.

[57]  Philippe Ravaud,et al.  Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review , 2013, BMJ.

[58]  R Gray,et al.  Hysterectomy, endometrial destruction, and levonorgestrel releasing intrauterine system (Mirena) for heavy menstrual bleeding: systematic review and meta-analysis of data from individual patients , 2010, BMJ : British Medical Journal.

[59]  S D Walter,et al.  The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. , 1997, Journal of clinical epidemiology.

[60]  Douglas G Altman,et al.  Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews , 2009, BMJ : British Medical Journal.

[61]  Mei Lu,et al.  Comparative effectiveness research using matching‐adjusted indirect comparison: an application to treatment with guanfacine extended release or atomoxetine in children with attention‐deficit/hyperactivity disorder and comorbid oppositional defiant disorder , 2012, Pharmacoepidemiology and drug safety.

[62]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[63]  G. Lu,et al.  Assessing Evidence Inconsistency in Mixed Treatment Comparisons , 2006 .

[64]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[65]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[66]  Deepak L. Bhatt,et al.  Short- versus long-term dual antiplatelet therapy after drug-eluting stent implantation: an individual patient data pairwise and network meta-analysis. , 2015, Journal of the American College of Cardiology.

[67]  Mark Simmonds,et al.  A decade of individual participant data meta-analyses: A review of current practice. , 2015, Contemporary clinical trials.

[68]  Multiple treatment comparisons in a series of anti-malarial trials with an ordinal primary outcome and repeated treatment evaluations , 2012, Malaria Journal.

[69]  J. Reginster,et al.  Indirect comparison of bazedoxifene vs oral bisphosphonates for the prevention of vertebral fractures in postmenopausal osteoporotic women , 2014, Current medical research and opinion.

[70]  L. Poston,et al.  University of Birmingham Study protocol : differential effects of diet and physical activity based interventions in pregnancy on maternal and fetal outcomes-individual patient data (IPD) meta-analysis and health economic evaluation , 2014 .