Graphical augmentations to the funnel plot assess the impact of additional evidence on a meta-analysis.

OBJECTIVE We aim to illustrate the potential impact of a new study on a meta-analysis, which gives an indication of the robustness of the meta-analysis. STUDY DESIGN AND SETTING A number of augmentations are proposed to one of the most widely used of graphical displays, the funnel plot. Namely, 1) statistical significance contours, which define regions of the funnel plot in which a new study would have to be located to change the statistical significance of the meta-analysis; and 2) heterogeneity contours, which show how a new study would affect the extent of heterogeneity in a given meta-analysis. Several other features are also described, and the use of multiple features simultaneously is considered. RESULTS The statistical significance contours suggest that one additional study, no matter how large, may have a very limited impact on the statistical significance of a meta-analysis. The heterogeneity contours illustrate that one outlying study can increase the level of heterogeneity dramatically. CONCLUSION The additional features of the funnel plot have applications including 1) informing sample size calculations for the design of future studies eligible for inclusion in the meta-analysis; and 2) informing the updating prioritization of a portfolio of meta-analyses such as those prepared by the Cochrane Collaboration.

[1]  Jun Shen,et al.  Association between Fecal Bile Acids and Colorectal Cancer: A Meta-analysis of Observational Studies , 2008, Yonsei medical journal.

[2]  Nicola J Cooper,et al.  Evidence synthesis as the key to more coherent and efficient research , 2009, BMC medical research methodology.

[3]  Julian P T Higgins,et al.  Graphical displays for meta‐analysis: An overview with suggestions for practice , 2010, Research synthesis methods.

[4]  J. Fleetham,et al.  Oral appliances for obstructive sleep apnoea. , 2006, The Cochrane database of systematic reviews.

[5]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[6]  Dong Zhou,et al.  Sanchi for acute ischaemic stroke. , 2008, The Cochrane database of systematic reviews.

[7]  Jonathan A C Sterne,et al.  The Funnel Plot , 2006 .

[8]  Chandler Stolp,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[9]  Alex J Sutton,et al.  An encouraging assessment of methods to inform priorities for updating systematic reviews. , 2009, Journal of clinical epidemiology.

[10]  J. Fleetham,et al.  Oral appliances for obstructive sleep apnoea. , 2003, The Cochrane database of systematic reviews.

[11]  P Hajek,et al.  Aversive smoking for smoking cessation. , 2001, The Cochrane database of systematic reviews.

[12]  Alex J Sutton,et al.  Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. , 2008, Journal of clinical epidemiology.

[13]  Alexander J Sutton,et al.  Empirical assessment suggests that existing evidence could be used more fully in designing randomized controlled trials. , 2010, Journal of clinical epidemiology.

[14]  L M Bouter,et al.  Muscle relaxants for non-specific low back pain. , 2003, The Cochrane database of systematic reviews.

[15]  M. Borenstein,et al.  Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments , 2006 .

[16]  Jonathan A C Sterne,et al.  Funnel Plots in Meta-analysis , 2004 .

[17]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[18]  G. Guyatt,et al.  The Science of Reviewing Research a , 1993, Annals of the New York Academy of Sciences.

[19]  I. Olkin,et al.  The case of the misleading funnel plot , 2006, BMJ : British Medical Journal.

[20]  Max H Pittler,et al.  Kava extract versus placebo for treating anxiety , 2003 .

[21]  J. Sterne,et al.  Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. , 2001, Journal of clinical epidemiology.

[22]  Karel G M Moons,et al.  More than numbers: the power of graphs in meta-analysis. , 2008, American journal of epidemiology.

[23]  Judith J Prochaska,et al.  Opioid antagonists for smoking cessation. , 2013, The Cochrane database of systematic reviews.

[24]  Alexander Tsertsvadze,et al.  Systematic reviews: when is an update an update? , 2006, The Lancet.

[25]  J. Grimshaw,et al.  When and how to update systematic reviews. , 2008, The Cochrane database of systematic reviews.

[26]  Nicola J Cooper,et al.  Evidence‐based sample size calculations based upon updated meta‐analysis , 2007, Statistics in medicine.

[27]  Sabine Landau,et al.  Psychological interventions to improve glycaemic control in patients with type 1 diabetes: systematic review and meta-analysis of randomised controlled trials , 2006, BMJ : British Medical Journal.