Individual participant data meta‐analysis of continuous outcomes: A comparison of approaches for specifying and estimating one‐stage models

One‐stage individual participant data meta‐analysis models should account for within‐trial clustering, but it is currently debated how to do this. For continuous outcomes modeled using a linear regression framework, two competing approaches are a stratified intercept or a random intercept. The stratified approach involves estimating a separate intercept term for each trial, whereas the random intercept approach assumes that trial intercepts are drawn from a normal distribution. Here, through an extensive simulation study for continuous outcomes, we evaluate the impact of using the stratified and random intercept approaches on statistical properties of the summary treatment effect estimate. Further aims are to compare (i) competing estimation options for the one‐stage models, including maximum likelihood and restricted maximum likelihood, and (ii) competing options for deriving confidence intervals (CI) for the summary treatment effect, including the standard normal‐based 95% CI, and more conservative approaches of Kenward‐Roger and Satterthwaite, which inflate CIs to account for uncertainty in variance estimates. The findings reveal that, for an individual participant data meta‐analysis of randomized trials with a 1:1 treatment:control allocation ratio and heterogeneity in the treatment effect, (i) bias and coverage of the summary treatment effect estimate are very similar when using stratified or random intercept models with restricted maximum likelihood, and thus either approach could be taken in practice, (ii) CIs are generally best derived using either a Kenward‐Roger or Satterthwaite correction, although occasionally overly conservative, and (iii) if maximum likelihood is required, a random intercept performs better than a stratified intercept model. An illustrative example is provided.

[1]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[2]  Mark Simmonds,et al.  A decade of individual participant data meta-analyses: A review of current practice. , 2015, Contemporary clinical trials.

[3]  Omar Paccagnella,et al.  Sample Size and Accuracy of Estimates in Multilevel Models , 2011 .

[4]  Richard D. Riley,et al.  Random effects meta‐analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation , 2016, Statistics in medicine.

[5]  John M. Ferron,et al.  How Low Can You Go? An Investigation of the Influence of Sample Size and Model Complexity on Point and Interval Estimates in Two-Level Linear Models , 2014 .

[6]  Paul Landais,et al.  Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. , 2004, Journal of clinical epidemiology.

[7]  Susan T. Hibbard,et al.  Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches , 2009, Behavior research methods.

[8]  R. Moineddin,et al.  A simulation study of sample size for multilevel logistic regression models , 2007, BMC medical research methodology.

[9]  Richard D Riley,et al.  Meta‐analysis using individual participant data: one‐stage and two‐stage approaches, and why they may differ , 2016, Statistics in medicine.

[10]  Richard D Riley,et al.  Interpretation of random effects meta-analyses , 2011, BMJ : British Medical Journal.

[11]  Kurex Sidik,et al.  A simple confidence interval for meta‐analysis , 2002, Statistics in medicine.

[12]  Daniel M. McNeish Estimation Methods for Mixed Logistic Models with Few Clusters , 2016, Multivariate behavioral research.

[13]  Ewout W. Steyerberg,et al.  Individual participant data meta-analyses should not ignore clustering , 2013, Journal of clinical epidemiology.

[14]  Laura M. Stapleton,et al.  The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration , 2014, Educational Psychology Review.

[15]  Mark C Simmonds,et al.  Meta-analysis of individual patient data from randomized trials: a review of methods used in practice , 2005, Clinical trials.

[16]  M. Kenward,et al.  Small sample inference for fixed effects from restricted maximum likelihood. , 1997, Biometrics.

[17]  François Gueyffier,et al.  Systolic and Diastolic Blood Pressure Lowering as Determinants of Cardiovascular Outcome , 2005, Hypertension.

[18]  G. B. Schaalje,et al.  Adequacy of approximations to distributions of test statistics in complex mixed linear models , 2002 .

[19]  F. E. Satterthwaite An approximate distribution of estimates of variance components. , 1946, Biometrics.

[20]  H C Van Houwelingen,et al.  A bivariate approach to meta-analysis. , 1993, Statistics in medicine.

[21]  Theo Stijnen,et al.  A comparison of seven random‐effects models for meta‐analyses that estimate the summary odds ratio , 2018, Statistics in medicine.

[22]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[23]  M. A. Best Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[24]  M. Parmar,et al.  Meta-analysis of the literature or of individual patient data: is there a difference? , 1993, The Lancet.

[25]  D. Spiegelhalter,et al.  Bayesian Approaches to Clinical Trials and Health-Care Evaluation: Spiegelhalter/Clinical Trials and Health-Care Evaluation , 2004 .

[26]  B. Kahan,et al.  Analysis of multicentre trials with continuous outcomes: when and how should we account for centre effects? , 2013, Statistics in medicine.

[27]  Richard D Riley,et al.  Meta‐analysis of randomised trials with a continuous outcome according to baseline imbalance and availability of individual participant data , 2013, Statistics in medicine.

[28]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[29]  Cynthia D Mulrow,et al.  Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change , 2014, Annals of Internal Medicine.

[30]  J. Hartung,et al.  A refined method for the meta‐analysis of controlled clinical trials with binary outcome , 2001, Statistics in medicine.

[31]  Stephen Senn,et al.  The Many Modes of Meta , 2000 .

[32]  Satterthwaite Fe An approximate distribution of estimates of variance components. , 1946 .

[33]  James R Carpenter,et al.  Meta‐analysis of Gaussian individual patient data: Two‐stage or not two‐stage? , 2018, Statistics in medicine.

[34]  Louise J Jackson,et al.  Effects of antenatal diet and physical activity on maternal and fetal outcomes: individual patient data meta-analysis and health economic evaluation. , 2017, Health technology assessment.