Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis

We study recently developed nonparametric methods for estimating the number of missing studies that might exist in a meta-analysis and the effect that these studies might have had on its outcome. These are simple rank-based data augmentation techniques, which formalize the use of funnel plots. We show that they provide effective and relatively powerful tests for evaluating the existence of such publication bias. After adjusting for missing studies, we find that the point estimate of the overall effect size is approximately correct and coverage of the effect size confidence intervals is substantially improved, in many cases recovering the nominal confidence levels entirely. We illustrate the trim and fill method on existing meta-analyses of studies in clinical trials and psychometrics.

[1]  L. Bero,et al.  Publication bias and research on passive smoking: comparison of published and unpublished studies. , 1998, JAMA.

[2]  C. Begg,et al.  Operating characteristics of a rank correlation test for publication bias. , 1994, Biometrics.

[3]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[4]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[5]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[6]  B J Biggerstaff,et al.  Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. , 1997, Statistics in medicine.

[7]  T. Gibson.  The Editor Regrets. , 1979, Journal of the Royal College of Physicians of London.

[8]  C. Begg,et al.  Publication bias : a problem in interpreting medical data , 1988 .

[9]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[10]  Colin B. Begg,et al.  An Approach for Assessing Publication Bias Prior to Performing a Meta-Analysis , 1992 .

[11]  Kerrie Mengersen,et al.  THE IMPACT OF METHOD CHOICE ON META‐ANALYSIS , 1995 .

[12]  Stephen W. Raudenbush,et al.  Magnitude of teacher expectancy effects on pupil IQ as a function of the credibility of expectancy induction: A synthesis of findings from 18 experiments. , 1984 .

[13]  L. Hedges.  Modeling publication selection effects in meta-analysis , 1992 .

[14]  I. Olkin,et al.  Models for estimating the number of unpublished studies. , 1996, Statistics in medicine.

[15]  K. Dickersin,et al.  Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. , 1992, JAMA.

[16]  P. Sleight,et al.  Publication bias , 1991, The Lancet.

[17]  S. Pocock,et al.  Can meta-analyses be trusted? , 1991, The Lancet.

[18]  Frederick Mosteller,et al.  Some Progress and Problems in Meta-Analysis of Clinical Trials , 1992 .

[19]  G. Smith,et al.  Bias in meta-analysis detected by a simple, graphical test , 1997, BMJ.

[20]  Jay Brand,et al.  File Drawer Problem , 2022, The SAGE Encyclopedia of Research Design.

[21]  R. Tweedie,et al.  Publication Bias in Meta-Analysis: A Bayesian Data-Augmentation Approach to Account for Issues Exempli(cid:12)ed in the Passive Smoking Debate , 1997 .

[22]  Joel B. Greenhouse,et al.  Selection Models and the File Drawer Problem , 1988 .

[23]  Richard L. Tweedie,et al.  A Non-parametric "Trim and Fill" Method of Assessing Publication Bias in Meta-analysis , 1998 .

[24]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[25]  D. P. Gaver,et al.  Combining Information: Statistical Issues and Opportunities for Research , 1993 .