Permutation inference methods for multivariate meta‐analysis

Multivariate meta-analysis is gaining prominence in evidence synthesis research because it enables simultaneous synthesis of multiple correlated outcome data, and random-effects models have generally been used for addressing between-studies heterogeneities. However, coverage probabilities of confidence regions or intervals for standard inference methods for random-effects models (e.g., restricted maximum likelihood estimation) cannot retain their nominal confidence levels in general, especially when the number of synthesized studies is small because their validities depend on large sample approximations. In this article, we provide permutation-based inference methods that enable exact joint inferences for average outcome measures without large sample approximations. We also provide accurate marginal inference methods under general settings of multivariate meta-analyses. We propose effective approaches for permutation inferences using optimal weighting based on the efficient score statistic. The effectiveness of the proposed methods is illustrated via applications to bivariate meta-analyses of diagnostic accuracy studies for airway eosinophilia in asthma and a network meta-analysis for antihypertensive drugs on incident diabetes, as well as through simulation experiments. In numerical evaluations performed via simulations, our methods generally provided accurate confidence regions or intervals under a broad range of settings, whereas the current standard inference methods exhibited serious undercoverage properties. This article is protected by copyright. All rights reserved.

[1]  Jean-Marie Dufour,et al.  Monte Carlo tests with nuisance parameters: a general approach to finite-sample inference and nonstandard , 2006 .

[2]  Hisashi Noma,et al.  Bartlett‐type corrections and bootstrap adjustments of likelihood‐based inference methods for network meta‐analysis , 2018, Statistics in medicine.

[3]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[4]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[5]  D A Follmann,et al.  Valid Inference in Random Effects Meta‐Analysis , 1999, Biometrics.

[6]  Jérémie F. Cohen,et al.  Diagnostic accuracy of minimally invasive markers for detection of airway eosinophilia in asthma: a systematic review and meta-analysis. , 2015, The Lancet. Respiratory medicine.

[7]  A Whitehead,et al.  Borrowing strength from external trials in a meta-analysis. , 1996, Statistics in medicine.

[8]  Jean-Marie Dufour,et al.  Monte Carlo Test Methods in Econometrics , 2007 .

[9]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[10]  J. J. Higgins,et al.  Effect of tail weight and outliers on power and type-i error of robust permutation tests for location , 1987 .

[11]  Dan Jackson,et al.  Multivariate meta-analysis: Potential and promise , 2011, Statistics in medicine.

[12]  G. Molenberghs,et al.  Linear Mixed Models for Longitudinal Data , 2001 .

[13]  Georgia Salanti,et al.  Indirect and mixed‐treatment comparison, network, or multiple‐treatments meta‐analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool , 2012, Research synthesis methods.

[14]  Oliver Kuss,et al.  Methods to calculate uncertainty in the estimated overall effect size from a random‐effects meta‐analysis , 2018, Research synthesis methods.

[15]  Patrick Bossuyt,et al.  Systematic Reviews of Diagnostic Test Accuracy , 2008, Annals of Internal Medicine.

[16]  Daniel Jackson,et al.  A refined method for multivariate meta-analysis and meta-regression , 2013, Statistics in medicine.

[17]  D. Cox,et al.  Parameter Orthogonality and Approximate Conditional Inference , 1987 .

[18]  Min‐ge Xie,et al.  Exact inference on meta-analysis with generalized fixed-effects and random-effects models , 2018 .

[19]  Hisashi Noma,et al.  Confidence intervals for a random‐effects meta‐analysis based on Bartlett‐type corrections , 2011, Statistics in medicine.

[20]  Jonathan J Deeks,et al.  Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. , 2001, BMJ.

[21]  A. Owen,et al.  Incident diabetes in clinical trials of antihypertensive drugs , 2007, The Lancet.

[22]  Dan Jackson,et al.  Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression‡ , 2012, Research synthesis methods.

[23]  Bas Donkers,et al.  A Method of Moments Estimator for Semiparametric Index Models , 2005 .

[24]  Dan Jackson,et al.  Extending DerSimonian and Laird's methodology to perform multivariate random effects meta‐analyses , 2009, Statistics in medicine.

[25]  Richard D Riley,et al.  A matrix‐based method of moments for fitting multivariate network meta‐analysis models with multiple outcomes and random inconsistency effects , 2017, Biometrics.

[26]  AE Ades,et al.  Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies‡ , 2012, Research synthesis methods.

[27]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[28]  Richard D Riley,et al.  Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples , 2017, British Medical Journal.

[29]  G. Grégoire,et al.  Discrepancies between meta-analyses and subsequent large randomized, controlled trials. , 1997, The New England journal of medicine.

[30]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[31]  Richard D Riley,et al.  A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression , 2013, Biometrical journal. Biometrische Zeitschrift.

[32]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[33]  Sarah E. Brockwell,et al.  A comparison of statistical methods for meta‐analysis , 2001, Statistics in medicine.

[34]  Douglas G Altman,et al.  Systematic reviews of evaluations of prognostic variables , 2001, BMJ : British Medical Journal.

[35]  Josée Dupuis,et al.  A Method of Moments Estimator for Random Effect Multivariate Meta‐Analysis , 2012, Biometrics.