Deriving percentage study weights in multi-parameter meta-analysis models: with application to meta-regression, network meta-analysis and one-stage individual participant data models

Many meta-analysis models contain multiple parameters, for example due to multiple outcomes, multiple treatments or multiple regression coefficients. In particular, meta-regression models may contain multiple study-level covariates, and one-stage individual participant data meta-analysis models may contain multiple patient-level covariates and interactions. Here, we propose how to derive percentage study weights for such situations, in order to reveal the (otherwise hidden) contribution of each study toward the parameter estimates of interest. We assume that studies are independent, and utilise a decomposition of Fisher’s information matrix to decompose the total variance matrix of parameter estimates into study-specific contributions, from which percentage weights are derived. This approach generalises how percentage weights are calculated in a traditional, single parameter meta-analysis model. Application is made to one- and two-stage individual participant data meta-analyses, meta-regression and network (multivariate) meta-analysis of multiple treatments. These reveal percentage study weights toward clinically important estimates, such as summary treatment effects and treatment-covariate interactions, and are especially useful when some studies are potential outliers or at high risk of bias. We also derive percentage study weights toward methodologically interesting measures, such as the magnitude of ecological bias (difference between within-study and across-study associations) and the amount of inconsistency (difference between direct and indirect evidence in a network meta-analysis).

[1]  C D Naylor,et al.  Meta-analysis of controlled clinical trials. , 1989, The Journal of rheumatology.

[2]  Dan Jackson,et al.  Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression‡ , 2012, Research synthesis methods.

[3]  D. A. Williams,et al.  Generalized Linear Model Diagnostics Using the Deviance and Single Case Deletions , 1987 .

[4]  D. Pregibon.  Logistic Regression Diagnostics , 1981 .

[5]  Richard D Riley,et al.  Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. , 2007, Journal of clinical epidemiology.

[6]  Georgia Salanti,et al.  Evaluation of networks of randomized trials , 2008, Statistical methods in medical research.

[7]  Wolfgang Viechtbauer,et al.  Two new methods to fit models for network meta-analysis with random inconsistency effects , 2016, BMC Medical Research Methodology.

[8]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[9]  Douglas G. Altman,et al.  Statistical Analysis of Individual Participant Data Meta-Analyses: A Comparison of Methods and Recommendations for Practice , 2012, PloS one.

[10]  R. Wolfinger,et al.  Generalized linear mixed models a pseudo-likelihood approach , 1993 .

[11]  Theo Stijnen,et al.  Advanced methods in meta‐analysis: multivariate approach and meta‐regression , 2002, Statistics in medicine.

[12]  I Olkin,et al.  Comparison of effect estimates from a meta-analysis of summary data from published studies and from a meta-analysis using individual patient data for ovarian cancer studies. , 1997, American journal of epidemiology.

[13]  D. Clayton,et al.  Statistical Models in Epidemiology , 1993 .

[14]  P C Lambert,et al.  A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. , 2002, Journal of clinical epidemiology.

[15]  Richard D Riley,et al.  Individual patient data meta-analysis of survival data using Poisson regression models , 2012, BMC Medical Research Methodology.

[16]  Evangelos Kontopantelis,et al.  A Short Guide and a Forest Plot Command (Ipdforest) for One-Stage Meta-Analysis , 2012 .

[17]  A J Sutton,et al.  Meta‐analysis of individual‐ and aggregate‐level data , 2008, Statistics in medicine.

[18]  Richard D Riley,et al.  Meta‐analysis of a binary outcome using individual participant data and aggregate data , 2010, Research synthesis methods.

[19]  Richard D Riley,et al.  Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data: The PRISMA-IPD Statement , 2015 .

[20]  C S Berkey,et al.  A random-effects regression model for meta-analysis. , 1995, Statistics in medicine.

[21]  Theo Stijnen,et al.  Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data , 2010, Statistics in medicine.

[22]  Sylvia Richardson,et al.  Improving ecological inference using individual‐level data , 2006, Statistics in medicine.

[23]  Richard D Riley,et al.  Meta‐analysis of randomised trials with a continuous outcome according to baseline imbalance and availability of individual participant data , 2013, Statistics in medicine.

[24]  Dan Jackson,et al.  A design-by-treatment interaction model for network meta-analysis with random inconsistency effects , 2014, Statistics in medicine.

[25]  G. Lu,et al.  Assessing Evidence Inconsistency in Mixed Treatment Comparisons , 2006 .

[26]  Jack Bowden,et al.  Individual patient data meta‐analysis of time‐to‐event outcomes: one‐stage versus two‐stage approaches for estimating the hazard ratio under a random effects model , 2011, Research synthesis methods.

[27]  L F Burmeister,et al.  A comparison of meta-analytic results using literature vs individual patient data. Paternal cell immunization for recurrent miscarriage. , 1995, JAMA.

[28]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[29]  Ian R. White,et al.  Multivariate Random-effects Meta-regression: Updates to Mvmeta , 2011 .

[30]  Richard D Riley,et al.  Borrowing of strength and study weights in multivariate and network meta-analysis , 2017, Statistical methods in medical research.

[31]  AE Ades,et al.  Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies‡ , 2012, Research synthesis methods.

[32]  B J Biggerstaff,et al.  Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. , 1997, Statistics in medicine.

[33]  Cynthia D Mulrow,et al.  Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change , 2014, Annals of Internal Medicine.

[34]  J F Tierney,et al.  A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners. , 2011, Journal of clinical epidemiology.

[35]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[36]  Helen Brown,et al.  Applied Mixed Models in Medicine , 2000, Technometrics.

[37]  Mark Simmonds,et al.  A decade of individual participant data meta-analyses: A review of current practice. , 2015, Contemporary clinical trials.

[38]  A Whitehead,et al.  Meta‐analysis of continuous outcome data from individual patients , 2001, Statistics in medicine.

[39]  Julian P. T. Higgins,et al.  Extending DerSimonian and Laird's methodology to perform network meta‐analyses with random inconsistency effects , 2015, Statistics in medicine.

[40]  Richard D Riley,et al.  Meta‐analysis of continuous outcomes combining individual patient data and aggregate data , 2008, Statistics in medicine.

[41]  Thomas Mathew,et al.  Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data , 2010, Biometrical journal. Biometrische Zeitschrift.

[42]  R. Littell.  SAS System for Mixed Models , 1996 .

[43]  Ian R. White,et al.  Network Meta-analysis , 2015 .

[44]  Orestis Efthimiou,et al.  Get real in individual participant data (IPD) meta‐analysis: a review of the methodology , 2015, Research synthesis methods.

[45]  Elizabeth Tipton,et al.  Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[46]  Theo Stijnen,et al.  The binomial distribution of meta-analysis was preferred to model within-study variability. , 2008, Journal of clinical epidemiology.

[47]  François Gueyffier,et al.  Systolic and Diastolic Blood Pressure Lowering as Determinants of Cardiovascular Outcome , 2005, Hypertension.

[48]  Kurex Sidik,et al.  Robust variance estimation for random effects meta-analysis , 2006, Comput. Stat. Data Anal..

[49]  Richard D Riley,et al.  Rejoinder to commentaries on ‘Multivariate meta‐analysis: Potential and promise’ , 2011 .

[50]  J. Hartung,et al.  A refined method for the meta‐analysis of controlled clinical trials with binary outcome , 2001, Statistics in medicine.

[51]  I Olkin,et al.  Comparison of meta-analysis versus analysis of variance of individual patient data. , 1998, Biometrics.

[52]  A. Sutton,et al.  Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey , 2012, BMJ : British Medical Journal.

[53]  Dan Jackson,et al.  Multivariate meta-analysis: Potential and promise , 2011, Statistics in medicine.

[54]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration , 2009, BMJ : British Medical Journal.