Meta‐analysis using individual participant data: one‐stage and two‐stage approaches, and why they may differ

Meta‐analysis using individual participant data (IPD) obtains and synthesises the raw, participant‐level data from a set of relevant studies. The IPD approach is becoming an increasingly popular tool as an alternative to traditional aggregate data meta‐analysis, especially as it avoids reliance on published results and provides an opportunity to investigate individual‐level interactions, such as treatment‐effect modifiers. There are two statistical approaches for conducting an IPD meta‐analysis: one‐stage and two‐stage. The one‐stage approach analyses the IPD from all studies simultaneously, for example, in a hierarchical regression model with random effects. The two‐stage approach derives aggregate data (such as effect estimates) in each study separately and then combines these in a traditional meta‐analysis model. There have been numerous comparisons of the one‐stage and two‐stage approaches via theoretical consideration, simulation and empirical examples, yet there remains confusion regarding when each approach should be adopted, and indeed why they may differ. In this tutorial paper, we outline the key statistical methods for one‐stage and two‐stage IPD meta‐analyses, and provide 10 key reasons why they may produce different summary results. We explain that most differences arise because of different modelling assumptions, rather than the choice of one‐stage or two‐stage itself. We illustrate the concepts with recently published IPD meta‐analyses, summarise key statistical software and provide recommendations for future IPD meta‐analyses. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

[1]  J F Tierney,et al.  A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners. , 2011, Journal of clinical epidemiology.

[2]  Richard D Riley,et al.  Interpretation of random effects meta-analyses , 2011, BMJ : British Medical Journal.

[3]  Matthieu Resche-Rigon,et al.  Multiple imputation for handling systematically missing confounders in meta‐analysis of individual participant data , 2013, Statistics in medicine.

[4]  Richard D Riley,et al.  Individual patient data meta-analysis of survival data using Poisson regression models , 2012, BMC Medical Research Methodology.

[5]  Julian P. T. Higgins,et al.  Meta-Regression in Stata , 2008 .

[6]  A Whitehead,et al.  The effect of selegiline in the treatment of people with Alzheimer's disease: a meta‐analysis of published trials , 2002, International journal of geriatric psychiatry.

[7]  Nicky J Welton,et al.  Allowing for uncertainty due to missing data in meta‐analysis—Part 2: Hierarchical models , 2008, Statistics in medicine.

[8]  Theo Stijnen,et al.  Advanced methods in meta‐analysis: multivariate approach and meta‐regression , 2002, Statistics in medicine.

[9]  Richard D Riley,et al.  Meta‐analysis of continuous outcomes combining individual patient data and aggregate data , 2008, Statistics in medicine.

[10]  Evangelos Kontopantelis,et al.  Performance of statistical methods for meta-analysis when true study effects are non-normally distributed: A simulation study , 2012, Statistical methods in medical research.

[11]  Thomas Mathew,et al.  Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data , 2010, Biometrical journal. Biometrische Zeitschrift.

[12]  D J Sargent,et al.  A general framework for random effects survival analysis in the Cox proportional hazards setting. , 1998, Biometrics.

[13]  J. Hartung,et al.  On tests of the overall treatment effect in meta‐analysis with normally distributed responses , 2001, Statistics in medicine.

[14]  Julian P T Higgins,et al.  An empirical comparison of heterogeneity variance estimators in 12 894 meta‐analyses , 2015, Research synthesis methods.

[15]  Patrick M M Bossuyt,et al.  Differences between univariate and bivariate models for summarizing diagnostic accuracy may not be large. , 2009, Journal of clinical epidemiology.

[16]  I Olkin,et al.  Comparison of effect estimates from a meta-analysis of summary data from published studies and from a meta-analysis using individual patient data for ovarian cancer studies. , 1997, American journal of epidemiology.

[17]  Jack Bowden,et al.  Individual patient data meta‐analysis of time‐to‐event outcomes: one‐stage versus two‐stage approaches for estimating the hazard ratio under a random effects model , 2011, Research synthesis methods.

[18]  Ghada Mohammed Abdallah Abo-Zaid,et al.  Individual patient data meta-analysis of prognostic factor studies , 2011 .

[19]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[20]  Theo Stijnen,et al.  The binomial distribution of meta-analysis was preferred to model within-study variability. , 2008, Journal of clinical epidemiology.

[21]  A Whitehead,et al.  A general parametric approach to the meta-analysis of randomized clinical trials. , 1991, Statistics in medicine.

[22]  Thorsten M. Buzug,et al.  IR-04-067 Adaptive Speciation : Introduction , 2004 .

[23]  S Greenland,et al.  Bias in the one-step method for pooling study results. , 1990, Statistics in medicine.

[24]  Richard D Riley,et al.  Performance of methods for meta-analysis of diagnostic test accuracy with few studies or sparse data , 2015, Statistical methods in medical research.

[25]  Karel G M Moons,et al.  Imputation of systematically missing predictors in an individual participant data meta‐analysis: a generalized approach using MICE , 2015, Statistics in medicine.

[26]  Ewout W. Steyerberg,et al.  Individual participant data meta-analyses should not ignore clustering , 2013, Journal of clinical epidemiology.

[27]  Patrick Royston,et al.  A new strategy for meta‐analysis of continuous covariates in observational studies , 2011, Statistics in medicine.

[28]  A. Hoes,et al.  Empirical comparison of subgroup effects in conventional and individual patient data meta-analyses , 2008, International Journal of Technology Assessment in Health Care.

[29]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration , 2013 .

[30]  Richard D. Riley,et al.  Summarising and validating test accuracy results across multiple studies for use in clinical practice , 2015, Statistics in medicine.

[31]  Tx Station Stata Statistical Software: Release 7. , 2001 .

[32]  Anne Whitehead,et al.  Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials , 2009, Clinical trials.

[33]  Jonathan A C Sterne,et al.  Meta-Analysis in Stata: An Updated Collection from the Stata Journal , 2015 .

[34]  Richard D Riley,et al.  Multivariate meta‐analysis: the effect of ignoring within‐study correlation , 2009 .

[35]  Paul Landais,et al.  Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. , 2004, Journal of clinical epidemiology.

[36]  Henrik Holmberg,et al.  Generalized linear models with clustered data: Fixed and random effects models , 2011, Comput. Stat. Data Anal..

[37]  Richard D Riley,et al.  Individual participant data meta-analysis of prognostic factor studies: state of the art? , 2012, BMC Medical Research Methodology.

[38]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[39]  Raghu Kacker,et al.  Random-effects model for meta-analysis of clinical trials: an update. , 2007, Contemporary clinical trials.

[40]  P. Williamson,et al.  A comparison of methods for fixed effects meta-analysis of individual patient data with time to event outcomes , 2007, Clinical trials.

[41]  Maengseok Noh,et al.  REML estimation for binary data in GLMMs , 2007 .

[42]  Patrick Royston,et al.  Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer , 2004, Statistics in medicine.

[43]  Stephen Kaptoge,et al.  Statistical methods for the time-to-event analysis of individual participant data from multiple epidemiological studies , 2010, International journal of epidemiology.

[44]  Mike Clarke,et al.  Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use , 2015, PLoS medicine.

[45]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[46]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[47]  Karel G M Moons,et al.  A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta‐analysis , 2013, Statistics in medicine.

[48]  Douglas G. Altman,et al.  Statistical Analysis of Individual Participant Data Meta-Analyses: A Comparison of Methods and Recommendations for Practice , 2012, PloS one.

[49]  R. Riley,et al.  Meta-analysis of randomized phase II trials to inform subsequent phase III decisions , 2014, Trials.

[50]  M. Greenwood An Introduction to Medical Statistics , 1932, Nature.

[51]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[52]  Jonathan J Deeks,et al.  Much ado about nothing: a comparison of the performance of meta‐analytical methods with rare events , 2007, Statistics in medicine.

[53]  S. Senn Hans van Houwelingen and the Art of Summing up , 2010, Biometrical journal. Biometrische Zeitschrift.

[54]  Michele Tarsilla Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[55]  Kurex Sidik,et al.  On Constructing Confidence Intervals for a Standardized Mean Difference in Meta-analysis , 2003 .

[56]  F Gueyffier,et al.  Multivariate meta-analysis using individual participant data , 2014 .

[57]  T. Mathew,et al.  On the Equivalence of Meta‐Analysis Using Literature and Using Individual Patient Data , 1999, Biometrics.

[58]  J P Pignon,et al.  Random effects survival models gave a better understanding of heterogeneity in individual patient data meta-analyses. , 2005, Journal of clinical epidemiology.

[59]  Douglas G Altman,et al.  Meta-analysis, Simpson's paradox, and the number needed to treat , 2002, BMC medical research methodology.

[60]  David Fisher,et al.  Two-stage Individual Participant Data Meta-analysis and Generalized Forest Plots , 2015 .

[61]  H Goldstein,et al.  A multilevel model framework for meta-analysis of clinical trials with binary outcomes. , 2000, Statistics in medicine.

[62]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[63]  George F Borm,et al.  The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method , 2014, BMC Medical Research Methodology.

[64]  Richard D Riley,et al.  Borrowing of strength and study weights in multivariate and network meta-analysis , 2017, Statistical methods in medical research.

[65]  M. Kenward,et al.  Small sample inference for fixed effects from restricted maximum likelihood. , 1997, Biometrics.

[66]  Guido Knapp,et al.  Improved tests for a random effects meta‐regression with a single covariate , 2003, Statistics in medicine.

[67]  Richard D Riley,et al.  Rejoinder to commentaries on ‘Multivariate meta‐analysis: Potential and promise’ , 2011 .

[68]  Ralf Bender,et al.  Methods to estimate the between‐study variance and its uncertainty in meta‐analysis† , 2015, Research synthesis methods.

[69]  J. Hartung,et al.  A refined method for the meta‐analysis of controlled clinical trials with binary outcome , 2001, Statistics in medicine.

[70]  I Olkin,et al.  Comparison of meta-analysis versus analysis of variance of individual patient data. , 1998, Biometrics.

[71]  P. Austin A Comparison of the Statistical Power of Different Methods for the Analysis of Repeated Cross-Sectional Cluster Randomization Trials with Binary Outcomes , 2010, The international journal of biostatistics.

[72]  Orestis Efthimiou,et al.  Get real in individual participant data (IPD) meta‐analysis: a review of the methodology , 2015, Research synthesis methods.

[73]  Harold I Feldman,et al.  Individual patient‐ versus group‐level data meta‐regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head , 2002, Statistics in medicine.

[74]  Mark C Simmonds,et al.  Meta-analysis of individual patient data from randomized trials: a review of methods used in practice , 2005, Clinical trials.

[75]  Kurex Sidik,et al.  A comparison of heterogeneity variance estimators in combining results of studies , 2007, Statistics in medicine.

[76]  Ingram Olkin,et al.  Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta‐analysis with zero cells , 2009, Statistics in medicine.

[77]  Douglas G. Altman,et al.  Metan: Fixed- and Random-Effects Meta-Analysis , 2008 .

[78]  Ian R. White,et al.  Multivariate Random-effects Meta-analysis , 2009 .

[79]  Stephen Senn,et al.  The Many Modes of Meta , 2000 .

[80]  D. Grobbee,et al.  Comparison of methods of handling missing data in individual patient data meta-analyses: an empirical example on antibiotics in children with acute otitis media. , 2007, American journal of epidemiology.

[81]  G. Zou,et al.  A modified poisson regression approach to prospective studies with binary data. , 2004, American journal of epidemiology.

[82]  Paula R Williamson,et al.  Investigating heterogeneity in an individual patient data meta‐analysis of time to event outcomes , 2005, Statistics in medicine.

[83]  Stephen Burgess,et al.  Combining multiple imputation and meta-analysis with individual participant data , 2013, Statistics in medicine.

[84]  Evangelos Kontopantelis,et al.  Metaan: Random-effects Meta-analysis , 2010 .

[85]  Theo Stijnen,et al.  Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data , 2010, Statistics in medicine.

[86]  Joachim Hartung,et al.  An Alternative Method for Meta‐Analysis , 1999 .

[87]  R. Peto,et al.  Beta blockade during and after myocardial infarction: an overview of the randomized trials. , 1985, Progress in cardiovascular diseases.

[88]  Evangelos Kontopantelis,et al.  A Short Guide and a Forest Plot Command (Ipdforest) for One-Stage Meta-Analysis , 2012 .

[89]  Richard D Riley,et al.  Meta‐analysis of a binary outcome using individual participant data and aggregate data , 2010, Research synthesis methods.

[90]  Richard D Riley,et al.  Multilevel mixed effects parametric survival models using adaptive Gauss–Hermite quadrature with application to recurrent events and individual participant data meta‐analysis , 2014, Statistics in medicine.

[91]  C S Berkey,et al.  A random-effects regression model for meta-analysis. , 1995, Statistics in medicine.

[92]  Peter Dalgaard,et al.  R Development Core Team (2010): R: A language and environment for statistical computing , 2010 .

[93]  Kurex Sidik,et al.  A simple confidence interval for meta‐analysis , 2002, Statistics in medicine.

[94]  Mark Simmonds,et al.  A decade of individual participant data meta-analyses: A review of current practice. , 2015, Contemporary clinical trials.

[95]  Peter C Austin,et al.  Estimating Multilevel Logistic Regression Models When the Number of Clusters is Low: A Comparison of Different Statistical Software Procedures , 2010, The international journal of biostatistics.

[96]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1997, Statistics in medicine.

[97]  J. R. Carpenter,et al.  Multiple imputation for IPD meta‐analysis: allowing for heterogeneity and studies with missing covariates , 2015, Statistics in medicine.

[98]  D Zeng,et al.  Equity-specific effects of interventions to promote physical activity among middle-aged and older adults: results from applying a novel equity-specific re-analysis strategy , 2021, International Journal of Behavioral Nutrition and Physical Activity.

[99]  Dan Jackson,et al.  Multivariate meta-analysis: Potential and promise , 2011, Statistics in medicine.

[100]  Alexander J Sutton,et al.  What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. , 2004, Statistics in medicine.

[101]  Paula R Williamson,et al.  An overview of methods and empirical comparison of aggregate data and individual patient data results for investigating heterogeneity in meta-analysis of time-to-event outcomes. , 2005, Journal of Evaluation In Clinical Practice.