One‐stage individual participant data meta‐analysis models: estimation of treatment‐covariate interactions must avoid ecological bias by separating out within‐trial and across‐trial information

Stratified medicine utilizes individual‐level covariates that are associated with a differential treatment effect, also known as treatment‐covariate interactions. When multiple trials are available, meta‐analysis is used to help detect true treatment‐covariate interactions by combining their data. Meta‐regression of trial‐level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta‐analyses are preferable to examine interactions utilizing individual‐level information. However, one‐stage IPD models are often wrongly specified, such that interactions are based on amalgamating within‐ and across‐trial information. We compare, through simulations and an applied example, fixed‐effect and random‐effects models for a one‐stage IPD meta‐analysis of time‐to‐event data where the goal is to estimate a treatment‐covariate interaction. We show that it is crucial to centre patient‐level covariates by their mean value in each trial, in order to separate out within‐trial and across‐trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta‐analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is −0.011 (95% CI: −0.019 to −0.003; p = 0.004), and thus highly significant, when amalgamating within‐trial and across‐trial information. However, when separating within‐trial from across‐trial information, the interaction is −0.007 (95% CI: −0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta‐analysts should only use within‐trial information to examine individual predictors of treatment effect and that one‐stage IPD models should separate within‐trial from across‐trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

[1]  Richard D Riley,et al.  Meta‐analysis of diagnostic test studies using individual patient data and aggregate data , 2008, Statistics in medicine.

[2]  Sylvia Richardson,et al.  Improving ecological inference using individual‐level data , 2006, Statistics in medicine.

[3]  A. Marson,et al.  Carbamazepine versus valproate monotherapy for epilepsy. , 2000, The Cochrane database of systematic reviews.

[4]  J. Pignon,et al.  Investigating trial and treatment heterogeneity in an individual patient data meta‐analysis of survival data by means of the penalized maximum likelihood approach , 2008, Statistics in medicine.

[5]  Mario Cleves,et al.  An Introduction To Survival Analysis Using Stata Third | , 2012 .

[6]  Simon G Thompson,et al.  Can meta-analysis help target interventions at individuals most likely to benefit? , 2005, The Lancet.

[7]  Harlan M Krumholz,et al.  Why data sharing should be the expected norm , 2015, BMJ : British Medical Journal.

[8]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[9]  J F Tierney,et al.  A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners. , 2011, Journal of clinical epidemiology.

[10]  Richard D Riley,et al.  Individual patient data meta-analysis of survival data using Poisson regression models , 2012, BMC Medical Research Methodology.

[11]  Mark C Simmonds,et al.  Random‐effects meta‐analysis of time‐to‐event data using the expectation–maximisation algorithm and shrinkage estimators , 2013, Research synthesis methods.

[12]  Douglas G. Altman,et al.  Statistical Analysis of Individual Participant Data Meta-Analyses: A Comparison of Methods and Recommendations for Practice , 2012, PloS one.

[13]  Josemir W Sander,et al.  Standards for epidemiologic studies and surveillance of epilepsy , 2011, Epilepsia.

[14]  Richard D Riley,et al.  Prognosis research strategy (PROGRESS) 4: Stratified medicine research , 2013, BMJ : British Medical Journal.

[15]  S Greenland,et al.  Ecological bias, confounding, and effect modification. , 1989, International journal of epidemiology.

[16]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[17]  Theo Stijnen,et al.  The binomial distribution of meta-analysis was preferred to model within-study variability. , 2008, Journal of clinical epidemiology.

[18]  S. Thompson,et al.  How should meta‐regression analyses be undertaken and interpreted? , 2002, Statistics in medicine.

[19]  Richard D Riley,et al.  Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes , 2013, BMJ : British Medical Journal.

[20]  J. Kalbfleisch,et al.  Between- and within-cluster covariate effects in the analysis of clustered data. , 1998, Biometrics.

[21]  J. Palmgren,et al.  Estimation of Multivariate Frailty Models Using Penalized Partial Likelihood , 2000, Biometrics.

[22]  Orestis Efthimiou,et al.  Get real in individual participant data (IPD) meta‐analysis: a review of the methodology , 2015, Research synthesis methods.

[23]  Paula R Williamson,et al.  Statistical issues in the assessment of the evidence for an interaction between factors in epilepsy trials , 2002, Statistics in medicine.

[24]  Harold I Feldman,et al.  Individual patient‐ versus group‐level data meta‐regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head , 2002, Statistics in medicine.

[25]  Mark C Simmonds,et al.  Meta-analysis of individual patient data from randomized trials: a review of methods used in practice , 2005, Clinical trials.

[26]  T. Dwyer,et al.  A discussion of some statistical methods for separating within-pair associations from associations among all twins in research on fetal origins of disease. , 2005, Paediatric and perinatal epidemiology.

[27]  M C Simmonds,et al.  Covariate heterogeneity in meta‐analysis: Criteria for deciding between meta‐regression and individual patient data , 2007, Statistics in medicine.

[28]  Xin Sun,et al.  Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses , 2010, BMJ : British Medical Journal.

[29]  Mark C Simmonds,et al.  Meta‐analysis of time‐to‐event data: a comparison of two‐stage methods , 2011, Research synthesis methods.

[30]  P C Lambert,et al.  A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. , 2002, Journal of clinical epidemiology.

[31]  Mark Simmonds,et al.  A decade of individual participant data meta-analyses: A review of current practice. , 2015, Contemporary clinical trials.

[32]  A Whitehead,et al.  Meta‐analysis of continuous outcome data from individual patients , 2001, Statistics in medicine.

[33]  Patrick Royston,et al.  The design of simulation studies in medical statistics , 2006, Statistics in medicine.

[34]  Jack Bowden,et al.  Individual patient data meta‐analysis of time‐to‐event outcomes: one‐stage versus two‐stage approaches for estimating the hazard ratio under a random effects model , 2011, Research synthesis methods.

[35]  B. Leroux,et al.  Between-subject and within-subject Statistical Information in Dental Research , 2000, Journal of dental research.

[36]  M. Piccart,et al.  Delivering precision medicine in oncology today and in future-the promise and challenges of personalised cancer medicine: a position paper by the European Society for Medical Oncology (ESMO). , 2014, Annals of oncology : official journal of the European Society for Medical Oncology.

[37]  D J Sargent,et al.  A general framework for random effects survival analysis in the Cox proportional hazards setting. , 1998, Biometrics.

[38]  Richard D Riley,et al.  Meta‐analysis of continuous outcomes combining individual patient data and aggregate data , 2008, Statistics in medicine.

[39]  Thomas Mathew,et al.  Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data , 2010, Biometrical journal. Biometrische Zeitschrift.

[40]  C. Hudis.  Trastuzumab--mechanism of action and use in clinical practice. , 2007, The New England journal of medicine.

[41]  T. Peters,et al.  Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. , 2001, Health technology assessment.

[42]  Ewout W. Steyerberg,et al.  Individual participant data meta-analyses should not ignore clustering , 2013, Journal of clinical epidemiology.

[43]  Yulia V. Marchenko,et al.  An Introduction to Survival Analysis Using Stata, Third Edition , 2010 .

[44]  Theo Stijnen,et al.  Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data , 2010, Statistics in medicine.

[45]  J. Beyene,et al.  Phase 3 trials of stereotactic radiosurgery with or without whole-brain radiation therapy for 1 to 4 brain metastases: individual patient data meta-analysis. , 2015, International journal of radiation oncology, biology, physics.

[46]  Sylvie Chevret,et al.  Practical methodology of meta-analysis of individual patient data using a survival outcome. , 2008, Contemporary clinical trials.

[47]  Paula R Williamson,et al.  Investigating heterogeneity in an individual patient data meta‐analysis of time to event outcomes , 2005, Statistics in medicine.

[48]  S. Bierma-Zeinstra,et al.  Individual patient data meta-analysis of trials investigating the effectiveness of intra-articular glucocorticoid injections in patients with knee or hip osteoarthritis: an OA Trial Bank protocol for a systematic review , 2013, Systematic Reviews.

[49]  Christopher H Schmid,et al.  Summing up evidence: one answer is not always enough , 1998, The Lancet.

[50]  Michael K Parides,et al.  Separation of individual‐level and cluster‐level covariate effects in regression analysis of correlated data , 2003, Statistics in medicine.

[51]  Stephen Kaptoge,et al.  Statistical methods for the time-to-event analysis of individual participant data from multiple epidemiological studies , 2010, International journal of epidemiology.

[52]  A J Sutton,et al.  Meta‐analysis of individual‐ and aggregate‐level data , 2008, Statistics in medicine.

[53]  Richard D Riley,et al.  Meta‐analysis of a binary outcome using individual participant data and aggregate data , 2010, Research synthesis methods.

[54]  Richard D Riley,et al.  Multilevel mixed effects parametric survival models using adaptive Gauss–Hermite quadrature with application to recurrent events and individual participant data meta‐analysis , 2014, Statistics in medicine.

[55]  Jayne Tierney,et al.  Two-stage meta-analysis of survival data from individual participants using percentile ratios , 2012, Statistics in medicine.

[56]  Michael J Crowther,et al.  Simulating biologically plausible complex survival data , 2013, Statistics in medicine.

[57]  F Gueyffier,et al.  Multivariate meta-analysis using individual participant data , 2014 .