Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials

Funnel plots, and tests for funnel plot asymmetry, have been widely used to examine bias in the results of meta-analyses. Funnel plot asymmetry should not be equated with publication bias, because it has a number of other possible causes. This article describes how to interpret funnel plot asymmetry, recommends appropriate tests, and explains the implications for choice of meta-analysis model

[1]  J. Ioannidis.  Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[2]  L. Stewart,et al.  Time to publication for results of clinical trials. , 2007, The Cochrane database of systematic reviews.

[3]  G. Smith,et al.  Bias in meta-analysis detected by a simple, graphical test , 1997, BMJ.

[4]  A. Sutton,et al.  Comparison of two methods to detect publication bias in meta-analysis. , 2006, JAMA.

[5]  George Davey Smith,et al.  Misleading meta-analysis , 1995, BMJ.

[6]  Christopher H Schmid,et al.  In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. , 2005, Journal of clinical epidemiology.

[7]  J. Ioannidis,et al.  Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials? , 2008, Philosophy, ethics, and humanities in medicine : PEHM.

[8]  I. Olkin,et al.  The case of the misleading funnel plot , 2006, BMJ : British Medical Journal.

[9]  J. Sterne,et al.  Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. , 2000, Journal of clinical epidemiology.

[10]  J. Sterne,et al.  Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. , 2001, Journal of clinical epidemiology.

[11]  Douglas G Altman,et al.  The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews , 2010, BMJ : British Medical Journal.

[12]  John P A Ioannidis,et al.  Interpretation of tests of heterogeneity and bias in meta-analysis. , 2008, Journal of evaluation in clinical practice.

[13]  R Peto,et al.  Effects of intravenous magnesium in suspected acute myocardial infarction: overview of randomised trials. , 1991, BMJ.

[14]  Raymond C. Schneider,et al.  ISIS-4: A randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58 050 patients with suspected acute myocardial infarction , 1995, The Lancet.

[15]  Gerta Rücker,et al.  Arcsine test for publication bias in meta‐analyses with binary outcomes , 2008, Statistics in medicine.

[16]  Santiago G. Moreno,et al.  Assessment of regression-based methods to adjust for publication bias through a comprehensive simulation study , 2009, BMC medical research methodology.

[17]  Alex J Sutton,et al.  Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. , 2008, Journal of clinical epidemiology.

[18]  Jonathan J Deeks,et al.  The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. , 2005, Journal of clinical epidemiology.

[19]  G. Rücker,et al.  Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. , 2011, Biostatistics.

[20]  F. Song,et al.  Dissemination and publication of research findings: an updated review of related biases. , 2010, Health technology assessment.

[21]  S Greenland,et al.  Random-effects meta-analyses are not always conservative. , 1999, American journal of epidemiology.

[22]  A. Hrõbjartsson,et al.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. , 2004, JAMA.

[23]  Julian P T Higgins,et al.  Controlling the risk of spurious findings from meta‐regression , 2004, Statistics in medicine.

[24]  Thomas A Trikalinos,et al.  The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey , 2007, Canadian Medical Association Journal.

[25]  David Moher,et al.  Addressing Reporting Biases , 2008 .

[26]  Alex J. Sutton,et al.  Assessing publication bias in meta‐analyses in the presence of between‐study heterogeneity , 2010 .

[27]  L. Rubenstein,et al.  Comprehensive geriatric assessment: a meta-analysis of controlled trials , 1993, The Lancet.

[28]  R. Rosenthal,et al.  Selective publication of antidepressant trials and its influence on apparent efficacy. , 2008, The New England journal of medicine.

[29]  J. Ioannidis,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias , 2008, PloS one.

[30]  J. Sterne,et al.  How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. , 2003, Health technology assessment.

[31]  Roger M Harbord,et al.  A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints , 2006, Statistics in medicine.

[32]  E. Antman,et al.  Early administration of intravenous magnesium to high-risk patients with acute myocardial infarction in the Magnesium in Coronaries (MAGIC) Trial: a randomised controlled trial , 2002, The Lancet.

[33]  Nicola J Cooper,et al.  Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications , 2009, BMJ : British Medical Journal.

[34]  Jonathan AC Sterne,et al.  Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy , 2005, The Lancet.

[35]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[36]  J Chilcott,et al.  Avoiding and identifying errors in health technology assessment models: qualitative study and methodological review. , 2010, Health technology assessment.

[37]  Sally Hopewell,et al.  Publication bias in clinical trials due to statistical significance or direction of trial results. , 2009, The Cochrane database of systematic reviews.

[38]  C. Begg,et al.  Publication bias : a problem in interpreting medical data , 1988 .

[39]  M. Egger,et al.  Who benefits from medical interventions? , 1994, BMJ.

[40]  Douglas G Altman,et al.  Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study , 2008, BMJ : British Medical Journal.

[41]  D. Eyding,et al.  Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials , 2010, BMJ : British Medical Journal.