Exclusion of studies with no events in both arms in meta-analysis impacted the conclusions.

OBJECTIVES Classical meta-analyses routinely treated studies with no events in both arms non-informative and excluded them from analyses. This study assessed whether such studies contain information and have influence on the conclusions of meta-analyses. DESIGN and setting: We collected meta-analyses of binary outcomes with at least one study having no events in both arms from Cochrane systematic reviews (2003-2018). We used the generalized linear mixed model to reanalyze these meta-analyses by two approaches: one including studies with no events in both arms and one excluding such studies. The magnitude and direction of odds ratio (OR), p-value, and width of 95% confidence interval (CI) were compared. A simulation study was conducted to examine the robustness of results. RESULTS We identified 442 meta-analyses. In comparing paired meta-analyses that included studies with no events in both arms versus those not, 8 (1.80%) resulted in different directions on OR; 41 (9.28%) altered conclusions on statistical significance. Substantial changes occurred on p-value (55.66% increased, 44.12% decreased) and the width of 95% CI (50.68% inflated, 49.32% declined) when excluding studies with no events. Simulation study confirmed these findings CONCLUSIONS: Studies with no events in both arms are not necessarily non-informative. Excluding such studies may alter conclusions.

[1]  Douglas G. Altman,et al.  Measurement in Medicine: The Analysis of Method Comparison Studies , 1983 .

[2]  J. Szalai,et al.  The randomized placebo-phase design for clinical trials , 2001 .

[3]  Alexander J Sutton,et al.  What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. , 2004, Statistics in medicine.

[4]  A. Iorio,et al.  Impact of including or excluding both-armed zero-event studies on using standard meta-analysis methods for rare event outcome: a simulation study , 2016, BMJ Open.

[5]  Haitao Chu,et al.  Bivariate random effects models for meta-analysis of comparative studies with binary outcomes: Methods for the absolute risk difference and relative risk , 2012, Statistical methods in medical research.

[6]  Ingram Olkin,et al.  Why add anything to nothing? The arcsine difference as a measure of treatment effect in meta‐analysis with zero cells , 2009, Statistics in medicine.

[7]  Theo Stijnen,et al.  A comparison of seven random‐effects models for meta‐analyses that estimate the summary odds ratio , 2018, Statistics in medicine.

[8]  Hendrik Koffijberg,et al.  Individual Participant Data Meta-Analysis for a Binary Outcome: One-Stage or Two-Stage? , 2013, PloS one.

[9]  O Kuss,et al.  Statistical methods for meta‐analyses including information from studies without any events—add nothing to nothing and succeed nevertheless , 2015, Statistics in medicine.

[10]  Minge Xie,et al.  Exact Meta-Analysis Approach for Discrete Data and its Application to 2 × 2 Tables With Rare Events , 2014, Journal of the American Statistical Association.

[11]  Richard D Riley,et al.  Meta‐analysis using individual participant data: one‐stage and two‐stage approaches, and why they may differ , 2016, Statistics in medicine.

[12]  Guido Skipka,et al.  The Peto odds ratio viewed as a new effect measure , 2014, Statistics in medicine.

[13]  Kate Flemming,et al.  Cochrane Qualitative and Implementation Methods Group guidance series-paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. , 2017, Journal of clinical epidemiology.

[14]  Joseph Beyene,et al.  Inclusion of zero total event trials in meta-analyses maintains analytic consistency and incorporates all available data , 2007, BMC medical research methodology.

[15]  K R Abrams,et al.  Bayesian methods in meta-analysis and evidence synthesis. , 2001, Statistical methods in medical research.

[16]  Evangelos Kontopantelis,et al.  A comparison of one‐stage vs two‐stage individual patient data meta‐analysis methods: A simulation study , 2018, Research synthesis methods.

[17]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[18]  H. Zou,et al.  Real-world Performance of Meta-analysis Methods for Double-Zero-Event Studies with Dichotomous Outcomes Using the Cochrane Database of Systematic Reviews , 2019, Journal of General Internal Medicine.

[19]  H. Chu,et al.  Performance of Between-study Heterogeneity Measures in the Cochrane Library , 2018, Epidemiology.

[20]  Andrew Forbes,et al.  Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. , 2014, The Cochrane database of systematic reviews.

[21]  Bayesian Estimation and Testing in Random Effects Meta-Analysis of Rare Binary Adverse Events , 2016, Statistics in biopharmaceutical research.

[22]  Nicola J Cooper,et al.  Meta-analysis of rare and adverse event data , 2002, Expert review of pharmacoeconomics & outcomes research.

[23]  S. Cole,et al.  Empirical Comparison of Publication Bias Tests in Meta-Analysis , 2018, Journal of General Internal Medicine.

[24]  Byron C. Wallace,et al.  Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study , 2019, Journal of clinical epidemiology.