Get real in individual participant data (IPD) meta‐analysis: a review of the methodology

Individual participant data (IPD) meta‐analysis is an increasingly used approach for synthesizing and investigating treatment effect estimates. Over the past few years, numerous methods for conducting an IPD meta‐analysis (IPD‐MA) have been proposed, often making different assumptions and modeling choices while addressing a similar research question. We conducted a literature review to provide an overview of methods for performing an IPD‐MA using evidence from clinical trials or non‐randomized studies when investigating treatment efficacy. With this review, we aim to assist researchers in choosing the appropriate methods and provide recommendations on their implementation when planning and conducting an IPD‐MA. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.

[1]  J. Thompson,et al.  Use of Bayesian Multivariate Meta-Analysis to Estimate the HAQ for Mapping Onto the EQ-5D Questionnaire in Rheumatoid Arthritis , 2014, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[2]  R M Turner,et al.  Multilevel models for meta-analysis, and their application to absolute risk differences , 2001, Statistical methods in medical research.

[3]  Karel G M Moons,et al.  Imputation of systematically missing predictors in an individual participant data meta‐analysis: a generalized approach using MICE , 2015, Statistics in medicine.

[4]  T. Haines,et al.  Inconsistent results in meta-analyses for the prevention of falls are found between study-level data and patient-level data. , 2011, Journal of clinical epidemiology.

[5]  Julian P T Higgins,et al.  Recent developments in meta‐analysis , 2008, Statistics in medicine.

[6]  Ewout W. Steyerberg,et al.  Individual participant data meta-analyses should not ignore clustering , 2013, Journal of clinical epidemiology.

[7]  Eric Q. Wu,et al.  Comparative Effectiveness Without Head-to-Head Trials , 2012, PharmacoEconomics.

[8]  D. Rubin,et al.  Multiple Imputation for Nonresponse in Surveys , 1989 .

[9]  R. Fitzpatrick,et al.  Issues in methodological research: perspectives from researchers and commissioners. , 2001, Health technology assessment.

[10]  Sylvia Richardson,et al.  Improving ecological inference using individual‐level data , 2006, Statistics in medicine.

[11]  J. Pignon,et al.  Investigating trial and treatment heterogeneity in an individual patient data meta‐analysis of survival data by means of the penalized maximum likelihood approach , 2008, Statistics in medicine.

[12]  Su Golder,et al.  Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview , 2011, PLoS medicine.

[13]  T. Yamaguchi,et al.  Proportional hazards models with random effects to examine centre effects in multicentre cancer clinical trials , 2002, Statistical methods in medical research.

[14]  Nicky J Welton,et al.  Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves , 2012, BMC Medical Research Methodology.

[15]  A. Sutton,et al.  Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey , 2012, BMJ : British Medical Journal.

[16]  Richard D Riley,et al.  Meta‐analysis of randomised trials with a continuous outcome according to baseline imbalance and availability of individual participant data , 2013, Statistics in medicine.

[17]  F. Vaida,et al.  Proportional hazards model with random effects. , 2000, Statistics in medicine.

[18]  S. Richardson,et al.  Hierarchical related regression for combining aggregate and individual data in studies of socio‐economic disease risk factors , 2007 .

[19]  H. Goldstein,et al.  Meta‐analysis using multilevel models with an application to the study of class size effects , 2000 .

[20]  P. Royston,et al.  Flexible parametric proportional‐hazards and proportional‐odds models for censored survival data, with application to prognostic modelling and estimation of treatment effects , 2002, Statistics in medicine.

[21]  M. Meredith,et al.  Exploring the Relationship Between Surrogates and Clinical Outcomes: Analysis of Individual Patient Data vs. Meta-regression on Group-Level Summary Statistics , 2003, Journal of biopharmaceutical statistics.

[22]  Catherine P. Bradshaw,et al.  The use of propensity scores to assess the generalizability of results from randomized trials , 2011, Journal of the Royal Statistical Society. Series A,.

[23]  Richard D Riley,et al.  Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. , 2007, Journal of clinical epidemiology.

[24]  Stephen Kaptoge,et al.  Statistical methods for the time-to-event analysis of individual participant data from multiple epidemiological studies , 2010, International journal of epidemiology.

[25]  Andy H. Lee,et al.  Multi-level zero-inflated Poisson regression modelling of correlated count data with excess zeros , 2006, Statistical methods in medical research.

[26]  Alexander Thompson,et al.  Thinking big: large-scale collaborative research in observational epidemiology , 2009, European Journal of Epidemiology.

[27]  David R. Jones,et al.  Systematic reviews of trials and other studies. , 1998, Health technology assessment.

[28]  Lena Osterhagen,et al.  Multiple Imputation For Nonresponse In Surveys , 2016 .

[29]  D J Sargent,et al.  A general framework for random effects survival analysis in the Cox proportional hazards setting. , 1998, Biometrics.

[30]  Matthieu Resche-Rigon,et al.  Multiple imputation for handling systematically missing confounders in meta‐analysis of individual participant data , 2013, Statistics in medicine.

[31]  Richard D Riley,et al.  Individual patient data meta-analysis of survival data using Poisson regression models , 2012, BMC Medical Research Methodology.

[32]  Mats O Karlsson,et al.  A linearization approach for the model‐based analysis of combined aggregate and individual patient data , 2014, Statistics in medicine.

[33]  Colin B. Begg,et al.  Random Effects Models for Combining Results from Controlled and Uncontrolled Studies in a Meta-Analysis , 1994 .

[34]  M. Clarke,et al.  Systematic reviews using individual patient data: a map for the minefields? , 1998, Annals of oncology : official journal of the European Society for Medical Oncology.

[35]  T. Stijnen,et al.  Review: a gentle introduction to imputation of missing values. , 2006, Journal of clinical epidemiology.

[36]  T. Yamaguchi,et al.  Investigating centre effects in a multi-centre clinical trial of superficial bladder cancer. , 1999, Statistics in medicine.

[37]  R. Doughty,et al.  Understanding differences in results from literature-based and individual patient meta-analyses: an example from meta-analyses of observational data. , 2011, International journal of cardiology.

[38]  Jeroen P Jansen,et al.  Network meta‐analysis of individual and aggregate level data , 2012, Research synthesis methods.

[39]  J F Tierney,et al.  A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners. , 2011, Journal of clinical epidemiology.

[40]  L. Stewart,et al.  To IPD or not to IPD? , 2002, Evaluation & the health professions.

[41]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[42]  Joseph G Ibrahim,et al.  Bayesian inference for multivariate meta‐analysis Box–Cox transformation models for individual patient data with applications to evaluation of cholesterol‐lowering drugs , 2013, Statistics in medicine.

[43]  J. Schafer.  Multiple imputation: a primer , 1999, Statistical methods in medical research.

[44]  E. Chelimsky,et al.  Cross-design Synthesis: A New Form of Meta-analysis for Combining Results from Randomized Clinical Trials and Medical-practice Databases , 1993, International Journal of Technology Assessment in Health Care.

[45]  M Buyse,et al.  On the relationship between response to treatment and survival time. , 1996, Statistics in medicine.

[46]  A J Sutton,et al.  Meta‐analysis of individual‐ and aggregate‐level data , 2008, Statistics in medicine.

[47]  H Goldstein,et al.  A multilevel model framework for meta-analysis of clinical trials with binary outcomes. , 2000, Statistics in medicine.

[48]  Richard D Riley,et al.  Meta‐analysis of a binary outcome using individual participant data and aggregate data , 2010, Research synthesis methods.

[49]  Stuart Mealing,et al.  The use of individual patient-level data (IPD) to quantify the impact of pretreatment predictors of response to treatment in chronic hepatitis B patients , 2013, BMJ Open.

[50]  A. Hoes,et al.  Differences in interaction and subgroup-specific effects were observed between randomized and nonrandomized studies in three empirical examples. , 2013, Journal of clinical epidemiology.

[51]  Fang Chen,et al.  Use of historical control data for assessing treatment effects in clinical trials , 2014, Pharmaceutical statistics.

[52]  Nicola J Cooper,et al.  Evidence synthesis as the key to more coherent and efficient research , 2009, BMC medical research methodology.

[53]  Alex J. Sutton,et al.  Bayesian methods for the cross‐design synthesis of epidemiological and toxicological evidence , 2005 .

[54]  A. Sutton,et al.  Mixed treatment comparisons using aggregate and individual participant level data , 2012, Statistics in medicine.

[55]  Paul Landais,et al.  Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. , 2004, Journal of clinical epidemiology.

[56]  L. Stewart,et al.  Systematic Reviews: Obtaining data from randomised controlled trials: how much do we need for reliable and informative meta-analyses? , 1994, BMJ.

[57]  David R. Jones,et al.  Hierarchical models in generalized synthesis of evidence: an example based on studies of breast cancer screening. , 2000, Statistics in medicine.

[58]  Ewout W Steyerberg,et al.  Logistic random effects regression models: a comparison of statistical packages for binary and ordinal outcomes , 2011, BMC medical research methodology.

[59]  Ruth Salway,et al.  A statistical framework for ecological and aggregate studies , 2001 .

[60]  Recai M. Yucel,et al.  Multiple imputation inference for multivariate multilevel continuous data with ignorable non-response , 2008, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[61]  P. Royston,et al.  A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials , 2004, Statistics in medicine.

[62]  Hans-Peter Piepho,et al.  The Use of Two‐Way Linear Mixed Models in Multitreatment Meta‐Analysis , 2012, Biometrics.

[63]  Joseph C Cappelleri,et al.  Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. , 2014, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[64]  M C Simmonds,et al.  Covariate heterogeneity in meta‐analysis: Criteria for deciding between meta‐regression and individual patient data , 2007, Statistics in medicine.

[65]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[66]  J. Ioannidis,et al.  Comparison of evidence of treatment effects in randomized and nonrandomized studies. , 2001, JAMA.

[67]  Daniel T. Larose,et al.  Grouped random effects models for Bayesian meta-analysis. , 1997, Statistics in medicine.

[68]  P. Tugwell,et al.  Checklists of methodological issues for review authors to consider when including non‐randomized studies in systematic reviews , 2013, Research synthesis methods.

[69]  Diederick E Grobbee,et al.  A systematic review of analytical methods used to study subgroups in (individual patient data) meta-analyses. , 2007, Journal of clinical epidemiology.

[70]  M. Kogevinas,et al.  Meta-Analysis Of Results And Individual Patient Data In Epidemiologal Studies , 2004 .

[71]  I Olkin,et al.  Comparison of effect estimates from a meta-analysis of summary data from published studies and from a meta-analysis using individual patient data for ovarian cancer studies. , 1997, American journal of epidemiology.

[72]  P C Lambert,et al.  A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. , 2002, Journal of clinical epidemiology.

[73]  Catrin Tudur Smith,et al.  Combining individual patient data and aggregate data in mixed treatment comparison meta‐analysis: Individual patient data may be beneficial if only for a subset of trials , 2013, Statistics in medicine.

[74]  Harvey Goldstein,et al.  Multilevel modelling of medical data , 2002, Statistics in medicine.

[75]  P. Tugwell,et al.  An introduction to methodological issues when including non‐randomised studies in systematic reviews on the effects of interventions , 2013, Research synthesis methods.

[76]  Lesley A Stewart,et al.  Investigating patient exclusion bias in meta-analysis. , 2004, International journal of epidemiology.

[77]  Simon G Thompson,et al.  Issues relating to confounding and meta‐analysis when including non‐randomized studies in systematic reviews on the effects of interventions , 2013, Research synthesis methods.

[78]  Gary H Lyman,et al.  The strengths and limitations of meta-analyses based on aggregate data , 2005, BMC Medical Research Methodology.

[79]  A Whitehead,et al.  Meta‐analysis of ordinal outcomes using individual patient data , 2001, Statistics in medicine.

[80]  L. Stewart,et al.  A COMPARISON OF THE RESULTS OF CHECKED VERSUS UNCHECKED INDIVIDUAL PATIENT DATA META-ANALYSES , 2002, International Journal of Technology Assessment in Health Care.

[81]  D. Grobbee,et al.  Comparison of methods of handling missing data in individual patient data meta-analyses: an empirical example on antibiotics in children with acute otitis media. , 2007, American journal of epidemiology.

[82]  Jayne Tierney,et al.  Two-stage meta-analysis of survival data from individual participants using percentile ratios , 2012, Statistics in medicine.

[83]  D. Hall.  Zero‐Inflated Poisson and Binomial Regression with Random Effects: A Case Study , 2000, Biometrics.

[84]  Anne Whitehead,et al.  Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials , 2009, Clinical trials.

[85]  J. Pignon,et al.  Individual patient-versus literature-based meta-analysis of survival data: time to event and event rate at a particular time can make a difference, an example based on head and neck cancer. , 2001, Controlled clinical trials.

[86]  Andrea Benedetti,et al.  Systematic review of methods for individual patient data meta- analysis with binary outcomes , 2014, BMC Medical Research Methodology.

[87]  J. Ioannidis,et al.  Predictive modeling and heterogeneity of baseline risk in meta-analysis of individual patient data. , 2001, Journal of clinical epidemiology.

[88]  Jack Bowden,et al.  Individual patient data meta‐analysis of time‐to‐event outcomes: one‐stage versus two‐stage approaches for estimating the hazard ratio under a random effects model , 2011, Research synthesis methods.

[89]  A. Hoes,et al.  Empirical comparison of subgroup effects in conventional and individual patient data meta-analyses , 2008, International Journal of Technology Assessment in Health Care.

[90]  Mei Lu,et al.  Matching-adjusted indirect comparisons: a new tool for timely comparative effectiveness research. , 2012, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[91]  Catrin Tudur Smith,et al.  Assessing the consistency assumption by exploring treatment by covariate interactions in mixed treatment comparison meta‐analysis: individual patient‐level covariates versus aggregate trial‐level covariates , 2012, Statistics in medicine.

[92]  Recai M Yucel,et al.  Random covariances and mixed-effects models for imputing multivariate multilevel continuous data , 2011, Statistical modelling.

[93]  J. Concato,et al.  Randomized, controlled trials, observational studies, and the hierarchy of research designs. , 2000, The New England journal of medicine.

[94]  Theo Stijnen,et al.  Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data , 2010, Statistics in medicine.

[95]  Henrik Holmberg,et al.  Generalized linear models with clustered data: Fixed and random effects models , 2011, Comput. Stat. Data Anal..

[96]  P. Williamson,et al.  A comparison of methods for fixed effects meta-analysis of individual patient data with time to event outcomes , 2007, Clinical trials.

[97]  V T Farewell,et al.  One-stage parametric meta-analysis of time-to-event outcomes , 2010, Statistics in medicine.

[98]  J P Pignon,et al.  Random effects survival models gave a better understanding of heterogeneity in individual patient data meta-analyses. , 2005, Journal of clinical epidemiology.

[99]  C Daniel Mullins,et al.  A questionnaire to assess the relevance and credibility of observational studies to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. , 2014, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[100]  Nicky J Welton,et al.  Allowing for uncertainty due to missing data in meta‐analysis—Part 2: Hierarchical models , 2008, Statistics in medicine.

[101]  Richard D Riley,et al.  Meta‐analysis of continuous outcomes combining individual patient data and aggregate data , 2008, Statistics in medicine.

[102]  Thomas Mathew,et al.  Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data , 2010, Biometrical journal. Biometrische Zeitschrift.

[103]  Jonathan J Deeks,et al.  Issues relating to study design and risk of bias when including non‐randomized studies in systematic reviews on the effects of interventions , 2013, Research synthesis methods.

[104]  A Whitehead,et al.  Meta‐analysis of continuous outcome data from individual patients , 2001, Statistics in medicine.

[105]  C. Mcgilchrist,et al.  ML and REML estimation in survival analysis with time dependent correlated frailty. , 1998, Statistics in medicine.

[106]  Sylvie Chevret,et al.  Practical methodology of meta-analysis of individual patient data using a survival outcome. , 2008, Contemporary clinical trials.

[107]  Paula R Williamson,et al.  Investigating heterogeneity in an individual patient data meta‐analysis of time to event outcomes , 2005, Statistics in medicine.

[108]  Andy H. Lee,et al.  Zero‐inflated Poisson regression with random effects to evaluate an occupational injury prevention programme , 2001, Statistics in medicine.

[109]  Stephen Burgess,et al.  Combining multiple imputation and meta-analysis with individual participant data , 2013, Statistics in medicine.

[110]  Christopher H Schmid,et al.  Summing up evidence: one answer is not always enough , 1998, The Lancet.

[111]  Eloise E Kaizar.  Estimating treatment effect via simple cross design synthesis , 2011, Statistics in medicine.

[112]  Douglas G. Altman,et al.  Statistical Analysis of Individual Participant Data Meta-Analyses: A Comparison of Methods and Recommendations for Practice , 2012, PloS one.

[113]  Junichi Sakamoto,et al.  Individual patient-level and study-level meta-analysis for investigating modifiers of treatment effect. , 2004, Japanese journal of clinical oncology.

[114]  Carl van Walraven,et al.  Individual patient meta-analysis--rewards and challenges. , 2010, Journal of clinical epidemiology.

[115]  Paula R Williamson,et al.  An overview of methods and empirical comparison of aggregate data and individual patient data results for investigating heterogeneity in meta-analysis of time-to-event outcomes. , 2005, Journal of Evaluation In Clinical Practice.

[116]  Harold I Feldman,et al.  Individual patient‐ versus group‐level data meta‐regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head , 2002, Statistics in medicine.

[117]  Mark C Simmonds,et al.  Meta-analysis of individual patient data from randomized trials: a review of methods used in practice , 2005, Clinical trials.