A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis

Abstract Meta-analysis collects and synthesizes results from individual studies to estimate an overall effect size. If published studies are chosen, say through a literature review, then an inherent selection bias may arise, because, for example, studies may tend to be published more readily if they are statistically significant, or deemed to be more “interesting” in terms of the impact of their outcomes. We develop a simple rank-based data augmentation technique, formalizing the use of funnel plots, to estimate and adjust for the numbers and outcomes of missing studies. Several nonparametric estimators are proposed for the number of missing studies, and their properties are developed analytically and through simulations. We apply the method to simulated and epidemiological datasets and show that it is both effective and consistent with other criteria in the literature.

[1]  H. Handoll,et al.  Interventions for replacing missing teeth: different times for loading dental implants. , 2004, The Cochrane database of systematic reviews.

[2]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[3]  George Casella,et al.  Bayesian Estimation of the Number of Unseen Studies in a Meta-Analysis , 1999 .

[4]  L. Bero,et al.  Publication bias and research on passive smoking: comparison of published and unpublished studies. , 1998, JAMA.

[5]  R. Tweedie,et al.  Publication Bias in Meta-Analysis: A Bayesian Data-Augmentation Approach to Account for Issues Exempli(cid:12)ed in the Passive Smoking Debate , 1997 .

[6]  G. Smith,et al.  Bias in meta-analysis detected by a simple, graphical test , 1997, BMJ.

[7]  B J Biggerstaff,et al.  Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis. , 1997, Statistics in medicine.

[8]  George Casella,et al.  Estimating the Number of Unseen Studies in a Meta-Analysis , 1997 .

[9]  I. Olkin,et al.  Models for estimating the number of unpublished studies. , 1996, Statistics in medicine.

[10]  B J Biggerstaff,et al.  Bayesian meta-analysis, with application to studies of ETS and lung cancer. , 1996, Lung cancer.

[11]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[12]  Kerrie Mengersen,et al.  THE IMPACT OF METHOD CHOICE ON META‐ANALYSIS , 1995 .

[13]  C. Begg,et al.  Operating characteristics of a rank correlation test for publication bias. , 1994, Biometrics.

[14]  M Sugita,et al.  Publication probability of a study on odds ratio value circumstantial evidence for publication bias in medical study areas. , 1994, The Tokai journal of experimental and clinical medicine.

[15]  R. Hanka.  The Handbook of Research Synthesis , 1994 .

[16]  D. Strachan,et al.  The Aftermath of Stroke. The experience of patients and their families , 1993 .

[17]  D. P. Gaver,et al.  Combining Information: Statistical Issues and Opportunities for Research , 1993 .

[18]  M Sugita,et al.  Estimating a summarized odds ratio whilst eliminating publication bias in meta-analysis. , 1992, Japanese journal of clinical oncology.

[19]  L. Hedges.  Modeling publication selection effects in meta-analysis , 1992 .

[20]  Colin B. Begg,et al.  An Approach for Assessing Publication Bias Prior to Performing a Meta-Analysis , 1992 .

[21]  Frederick Mosteller,et al.  Some Progress and Problems in Meta-Analysis of Clinical Trials , 1992 .

[22]  K. Dickersin,et al.  Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. , 1992, JAMA.

[23]  P. Sleight,et al.  Publication bias , 1991, The Lancet.

[24]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[25]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[26]  R F Galbraith,et al.  A note on graphical presentation of estimated odds ratios from several clinical trials. , 1988, Statistics in medicine.

[27]  Joel B. Greenhouse,et al.  Selection Models and the File Drawer Problem , 1988 .

[28]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[29]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[30]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[31]  A. Gordon.  The editor regrets , 1980, British medical journal.

[32]  The editor regrets , 1980 .

[33]  L. A. Marascuilo,et al.  Nonparametric and Distribution-Free Methods for the Social Sciences , 1977 .