A comparison of seven random‐effects models for meta‐analyses that estimate the summary odds ratio

Comparative trials that report binary outcome data are commonly pooled in systematic reviews and meta‐analyses. This type of data can be presented as a series of 2‐by‐2 tables. The pooled odds ratio is often presented as the outcome of primary interest in the resulting meta‐analysis. We examine the use of 7 models for random‐effects meta‐analyses that have been proposed for this purpose. The first of these models is the conventional one that uses normal within‐study approximations and a 2‐stage approach. The other models are generalised linear mixed models that perform the analysis in 1 stage and have the potential to provide more accurate inference. We explore the implications of using these 7 models in the context of a Cochrane Review, and we also perform a simulation study. We conclude that generalised linear mixed models can result in better statistical inference than the conventional 2‐stage approach but also that this type of model presents issues and difficulties. These challenges include more demanding numerical methods and determining the best way to model study specific baseline risks. One possible approach for analysts is to specify a primary model prior to performing the systematic review but also to present the results using other models in a sensitivity analysis. Only one of the models that we investigate is found to perform poorly so that any of the other models could be considered for either the primary or the sensitivity analysis.

[1]  Wolfgang Viechtbauer,et al.  Bias and Efficiency of Meta-Analytic Variance Estimators in the Random-Effects Model , 2005 .

[2]  Jonathan J. Shuster,et al.  Low‐event‐rate meta‐analyses of clinical trials: implementing good practices , 2016, Statistics in medicine.

[3]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[4]  Jonathan J Deeks,et al.  Much ado about nothing: a comparison of the performance of meta‐analytical methods with rare events , 2007, Statistics in medicine.

[5]  Leena Choi,et al.  Elucidating the Foundations of Statistical Inference with 2 x 2 Tables , 2015, PloS one.

[6]  S. Senn.  Hans van Houwelingen and the Art of Summing up , 2010, Biometrical journal. Biometrische Zeitschrift.

[7]  Ralf Bender,et al.  Methods to estimate the between‐study variance and its uncertainty in meta‐analysis† , 2015, Research synthesis methods.

[8]  Theodore J. Thompson,et al.  A MODEL FOR INTERVAL‐CENSORED TUBERCULOSIS OUTBREAK DATA , 1997 .

[9]  Andrea Benedetti,et al.  A comparison of analytic approaches for individual patient data meta-analyses with binary outcomes , 2017, BMC Medical Research Methodology.

[10]  Sarah E Brockwell,et al.  A simple method for inference on an overall effect in meta‐analysis , 2007, Statistics in medicine.

[11]  O Kuss,et al.  Statistical methods for meta‐analyses including information from studies without any events—add nothing to nothing and succeed nevertheless , 2015, Statistics in medicine.

[12]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[13]  R. Jennrich,et al.  Unbalanced repeated-measures models with structured covariance matrices. , 1986, Biometrics.

[14]  Douglas G. Altman,et al.  Effect Measures for Meta‒Analysis of Trials with Binary Outcomes , 2008 .

[15]  Ingram Olkin,et al.  Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta‐analysis with zero cells , 2009, Statistics in medicine.

[16]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[17]  Kalliopi Mylona,et al.  Meta‐analysis of clinical trials with rare events , 2015, Biometrical journal. Biometrische Zeitschrift.

[18]  Toshiya Sato,et al.  BIAS IN THE PETO ONE-STEP ESTIMATOR FOR THE COMMON ODDS RATIO , 2005 .

[19]  A. E. Ades,et al.  Absolute or relative effects? Arm‐based synthesis of trial data , 2015, Research synthesis methods.

[20]  Jack Bowden,et al.  A re-evaluation of the ‘quantile approximation method’ for random effects meta-analysis , 2008, Statistics in medicine.

[21]  J. Hartung,et al.  A refined method for the meta‐analysis of controlled clinical trials with binary outcome , 2001, Statistics in medicine.

[22]  Hisashi Noma,et al.  Confidence intervals for a random‐effects meta‐analysis based on Bartlett‐type corrections , 2011, Statistics in medicine.

[23]  Mark C Simmonds,et al.  A general framework for the use of logistic regression models in meta-analysis , 2016, Statistical methods in medical research.

[24]  Theo Stijnen,et al.  Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data , 2010, Statistics in medicine.

[25]  R. Peto,et al.  Beta blockade during and after myocardial infarction: an overview of the randomized trials. , 1985, Progress in cardiovascular diseases.

[26]  H C Van Houwelingen,et al.  A bivariate approach to meta-analysis. , 1993, Statistics in medicine.

[27]  Rebecca M Turner,et al.  Characteristics of meta-analyses and their component studies in the Cochrane Database of Systematic Reviews: a cross-sectional, descriptive analysis , 2011, BMC medical research methodology.

[28]  Richard D Riley,et al.  Meta‐analysis using individual participant data: one‐stage and two‐stage approaches, and why they may differ , 2016, Statistics in medicine.

[29]  Ian R. White,et al.  Simsum: Analyses of Simulation Studies Including Monte Carlo Error , 2010 .

[30]  David C Hoaglin,et al.  Misunderstandings about Q and ‘Cochran's Q test' in meta‐analysis , 2016, Statistics in medicine.

[31]  Kurex Sidik,et al.  A comparison of heterogeneity variance estimators in combining results of studies , 2007, Statistics in medicine.

[32]  Rakesh Lodha,et al.  Antibiotics for preventing complications in children with measles. , 2008, The Cochrane database of systematic reviews.

[33]  H Goldstein,et al.  A multilevel model framework for meta-analysis of clinical trials with binary outcomes. , 2000, Statistics in medicine.

[34]  Dan Jackson,et al.  Extending DerSimonian and Laird's methodology to perform multivariate random effects meta‐analyses , 2009, Statistics in medicine.

[35]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[36]  D J Spiegelhalter,et al.  Bayesian approaches to random-effects meta-analysis: a comparative study. , 1995, Statistics in medicine.

[37]  Julian P. T. Higgins,et al.  Extending DerSimonian and Laird's methodology to perform network meta‐analyses with random inconsistency effects , 2015, Statistics in medicine.

[38]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[39]  Jack Bowden,et al.  A sensitivity analysis framework for the treatment effect measure used in the meta-analysis of comparative binary data from randomised controlled trials , 2012, Statistics in medicine.

[40]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[41]  Sarah E. Brockwell,et al.  A comparison of statistical methods for meta‐analysis , 2001, Statistics in medicine.

[42]  Bradley P Carlin,et al.  Rejoinder to the discussion of “a Bayesian missing data framework for generalized multiple outcome mixed treatment comparisons,” by S. Dias and A. E. Ades , 2016, Research synthesis methods.

[43]  A E Ades,et al.  The Interpretation of Random-Effects Meta-Analysis in Decision Models , 2005, Medical decision making : an international journal of the Society for Medical Decision Making.

[44]  M. Bracken,et al.  Clinically useful measures of effect in binary analyses of randomized trials. , 1994, Journal of clinical epidemiology.

[45]  Jonathan J Deeks,et al.  Issues in the selection of a summary statistic for meta‐analysis of clinical trials with binary outcomes , 2002, Statistics in medicine.