Publication bias and small-study effects magnified effectiveness of antipsychotics but their relative ranking remained invariant.

OBJECTIVES Publication bias (PB) may seriously compromise inferences from meta-analyses. The aim of this article was to assess the potential effect of small-study effects and PB on the recently estimated relative effectiveness and ranking of pharmacological treatments for schizophrenia. STUDY DESIGN AND SETTING We used a recently published network of 167 trials involving 36,871 patients and comparing the effectiveness of 15 antipsychotics and placebo. We used novel visual and statistical methods to explore if smaller trials are associated with larger treatment effects and a selection model to explore if the probability of trial publication is associated with the magnitude of effect. We conducted a network meta-analysis of the published evidence as our primary analysis and used a sensitivity analysis considering low, moderate, and severe selection bias (that corresponds to the number of unpublished trials) with an aim to evaluate robustness of point estimates and ranking. We explored whether placebo-controlled and head-to-head trials are associated with different levels of PB. RESULTS We found that small placebo-controlled trials exaggerated slightly the efficacy of antipsychotics, and PB was not unlikely in the evidence based on placebo-controlled trials; however, ranking of antipsychotics remained robust. CONCLUSION The total evidence comprises many head-to-head trials that do not appear to be prone to small-study effects or PB, and indirect evidence appears to "wash out" some of the biases in the placebo-controlled trials.

[1]  Nicky J Welton,et al.  Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study. , 2013, International journal of epidemiology.

[2]  Nicola J Cooper,et al.  Adjusting for publication biases across similar interventions performed well when compared with gold standard data. , 2011, Journal of clinical epidemiology.

[3]  Georgia Salanti,et al.  Evaluation of networks of randomized trials , 2008, Statistical methods in medical research.

[4]  J. Geddes,et al.  Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis , 2009, The Lancet.

[5]  Andrew Thomas,et al.  The BUGS project: Evolution, critique and future directions , 2009, Statistics in medicine.

[6]  G. Salanti,et al.  Exploring and accounting for publication bias in mental health: a brief overview of methods , 2014, Evidence-Based Mental Health.

[7]  Kristian Thorlund,et al.  The Quality of Reporting Methods and Results in Network Meta-Analyses: An Overview of Reviews and Suggestions for Improvement , 2014, PloS one.

[8]  Jonathan A C Sterne,et al.  Investigating and Dealing with Publication and Other Biases , 2008 .

[9]  Nicky J Welton,et al.  A selection model for accounting for publication bias in a full network meta‐analysis , 2014, Statistics in medicine.

[10]  Dimitris Mavridis,et al.  A fully Bayesian application of the Copas selection model for publication bias extended to network meta‐analysis , 2013, Statistics in medicine.

[11]  Gerta Rücker,et al.  Empirical evaluation showed that the Copas selection model provided a useful summary in 80% of meta-analyses. , 2009, Journal of clinical epidemiology.

[12]  Deborah M Caldwell,et al.  Simultaneous comparison of multiple treatments: combining direct and indirect evidence , 2005, BMJ : British Medical Journal.

[13]  Anna Chaimani,et al.  Evaluating the Quality of Evidence from a Network Meta-Analysis , 2014, PloS one.

[14]  Alex J Sutton,et al.  Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. , 2008, Journal of clinical epidemiology.

[15]  J. Ioannidis,et al.  Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials , 2011, BMJ : British Medical Journal.

[16]  Georgia Salanti,et al.  Indirect and mixed‐treatment comparison, network, or multiple‐treatments meta‐analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool , 2012, Research synthesis methods.

[17]  K R Abrams,et al.  Modelling publication bias in meta-analysis: a review. , 2000, Statistical methods in medical research.

[18]  E. Turner,et al.  Publication Bias in Antipsychotic Trials: An Analysis of Efficacy Comparing the Published Literature to the US Food and Drug Administration Database , 2012, PLoS medicine.

[19]  Panagiota Spyridonos,et al.  Graphical Tools for Network Meta-Analysis in STATA , 2013, PloS one.

[20]  Philippe Ravaud,et al.  Impact of Reporting Bias in Network Meta-Analysis of Antidepressant Placebo-Controlled Trials , 2012, PloS one.

[21]  Harald Binder,et al.  Visualizing the flow of evidence in network meta‐analysis and characterizing mixed treatment comparisons , 2013, Statistics in medicine.

[22]  R. Rosenthal,et al.  Selective publication of antidepressant trials and its influence on apparent efficacy. , 2008, The New England journal of medicine.

[23]  Dimitris Mavridis,et al.  Comparative efficacy and tolerability of 15 antipsychotic drugs in schizophrenia: a multiple-treatments meta-analysis , 2013, The Lancet.

[24]  Anna Chaimani,et al.  Using network meta‐analysis to evaluate the existence of small‐study effects in a network of interventions , 2012, Research synthesis methods.

[25]  Georgia Salanti,et al.  Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. , 2011, Journal of clinical epidemiology.

[26]  Harald Binder,et al.  A graphical tool for locating inconsistency in network meta-analyses , 2013, BMC Medical Research Methodology.

[27]  G. Lu,et al.  Combination of direct and indirect evidence in mixed treatment comparisons , 2004, Statistics in medicine.

[28]  Jonathan A C Sterne,et al.  Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. , 2001, BMJ.