Using network meta‐analysis to evaluate the existence of small‐study effects in a network of interventions

Suggested methods for exploring the presence of small-study effects in a meta-analysis and the possibility of publication bias are associated with important limitations. When a meta-analysis comprises only a few studies, funnel plots are difficult to interpret, and regression-based approaches to test and account for small-study effects have low power. Assuming that the cause of funnel plot asymmetry is likely to affect an entire research field rather than only a particular comparison of interventions, we suggest that network meta-regression is employed to account for small-study effects in a set of related meta-analyses. We present several possible models for the direction and distribution of small-study effects and we describe the methods by re-analysing two published networks. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  Nicola J Cooper,et al.  Adjusting for publication biases across similar interventions performed well when compared with gold standard data. , 2011, Journal of clinical epidemiology.

[2]  Gerta Rücker,et al.  Empirical evaluation showed that the Copas selection model provided a useful summary in 80% of meta-analyses. , 2009, Journal of clinical epidemiology.

[3]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[4]  Thomas Lumley,et al.  Health outcomes associated with various antihypertensive therapies used as first-line agents: a network meta-analysis. , 2003, JAMA.

[5]  Gerta Rücker,et al.  Empirical evaluation suggests Copas selection model preferable to trim-and-fill method for selection bias in meta-analysis. , 2010, Journal of clinical epidemiology.

[6]  N. Welton,et al.  Addressing between‐study heterogeneity and inconsistency in mixed treatment comparisons: Application to stroke prevention treatments in individuals with non‐rheumatic atrial fibrillation , 2009, Statistics in medicine.

[7]  S G Thompson,et al.  Analysing the relationship between treatment effect and underlying risk in meta-analysis: comparison and development of approaches. , 2000, Statistics in medicine.

[8]  Nicky J Welton,et al.  Study designs to detect sponsorship and other biases in systematic reviews. , 2010, Journal of clinical epidemiology.

[9]  Alex J. Sutton,et al.  Assessing publication bias in meta‐analyses in the presence of between‐study heterogeneity , 2010 .

[10]  Gerta Rücker,et al.  Detecting and adjusting for small‐study effects in meta‐analysis , 2011, Biometrical journal. Biometrische Zeitschrift.

[11]  Julian P T Higgins,et al.  A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. , 2009, Journal of clinical epidemiology.

[12]  Santiago G. Moreno,et al.  Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study , 2009, BMC medical research methodology.

[13]  A. Sutton,et al.  Comparison of two methods to detect publication bias in meta-analysis. , 2006, JAMA.

[14]  Alex J Sutton,et al.  Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. , 2008, Journal of clinical epidemiology.

[15]  Michele Tarsilla.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[16]  Gerta Rücker,et al.  Arcsine test for publication bias in meta‐analyses with binary outcomes , 2008, Statistics in medicine.

[17]  L. Lindholm,et al.  Metabolic outcome during 1 year in newly detected hypertensives: results of the Antihypertensive Treatment and Lipid Profile in a North of Sweden Efficacy Evaluation (ALPINE study) , 2003, Journal of hypertension.

[18]  Nicky J Welton,et al.  Evaluating novel agent effects in multiple‐treatments meta‐regression , 2010, Statistics in medicine.

[19]  I Harvey,et al.  Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions. , 2008, Journal of clinical epidemiology.

[20]  Roger M Harbord,et al.  A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints , 2006, Statistics in medicine.

[21]  William J. Elliott.  Incident diabetes in clinical trials of antihypertensive drugs: a network meta-analysis , 2008 .

[22]  S D Walter,et al.  The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. , 1997, Journal of clinical epidemiology.

[23]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[24]  S G Thompson,et al.  Investigating underlying risk as a source of heterogeneity in meta-analysis. , 1997, Statistics in medicine.

[25]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1999 .

[26]  Jian Qing Shi,et al.  Meta‐analysis and sensitivity analysis for multi‐arm trials with selection bias , 2011, Statistics in medicine.

[27]  J. Copas,et al.  Meta-analysis, funnel plots and sensitivity analysis. , 2000, Biostatistics.

[28]  J Q Shi,et al.  A sensitivity analysis for publication bias in systematic reviews , 2001, Statistical methods in medical research.

[29]  J. Sterne,et al.  Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. , 2000, Journal of clinical epidemiology.

[30]  Nicky J Welton,et al.  Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta‐analysis , 2010 .

[31]  Ingram Olkin,et al.  Adjusting for publication bias in the presence of heterogeneity , 2003, Statistics in medicine.

[32]  J. Geddes,et al.  Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis , 2009, The Lancet.

[33]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[34]  K R Abrams,et al.  Modelling publication bias in meta-analysis: a review. , 2000, Statistical methods in medical research.