Empirical assessment of effect of publication bias on meta-analyses

Abstract Objective: To assess the effect of publication bias on the results and conclusions of systematic reviews and meta-analyses. Design: Analysis of published meta-analyses by trim and fill method. Studies: 48 reviews in Cochrane Database of Systematic Reviews that considered a binary endpoint and contained 10 or more individual studies. Main outcome measures: Number of reviews with missing studies and effect on conclusions of meta-analyses. Results: The trim and fill fixed effects analysis method estimated that 26 (54%) of reviews had missing studies and in 10 the number missing was significant. The corresponding figures with a random effects model were 23 (48%) and eight. In four cases, statistical inferences regarding the effect of the intervention were changed after the overall estimate for publication bias was adjusted for. Conclusions: Publication or related biases were common within the sample of meta-analyses assessed. In most cases these biases did not affect the conclusions. Nevertheless, researchers should check routinely whether conclusions of systematic reviews are robust to possible non-random selection mechanisms.

[1]  H. Handoll,et al.  Interventions for replacing missing teeth: different times for loading dental implants. , 2004, The Cochrane database of systematic reviews.

[2]  T. Coats.  Pre-hospital Medicine: The Principles and Practice of Immediate Care. , 2000 .

[3]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[4]  J. Eagles,et al.  Psychiatric trainees' experiences of, and reactions to, patient suicide , 2000 .

[5]  T. Sheldon,et al.  Relation between hostility and coronary heart disease , 1999, British medical journal.

[6]  M. Callaham,et al.  Medical Editors Trial Amnesty. , 1998, Annals of emergency medicine.

[7]  Richard L. Tweedie,et al.  A Non-parametric "Trim and Fill" Method of Assessing Publication Bias in Meta-analysis , 1998 .

[8]  S Duval,et al.  Practical estimates of the effect of publication bias in meta- analysis , 1998 .

[9]  R. Kendell,et al.  Consultant psychiatrists who retired prematurely in 1995 and 1996 , 1997 .

[10]  R. Tweedie,et al.  Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate - Comments and rejoinders , 1997 .

[11]  Wayne B Jonas,et al.  Are the clinical effects of homoeopathy placebo effects? A meta-analysis of placebo-controlled trials , 1997, The Lancet.

[12]  R. Horton.  Medical Editors Trial Amnesty , 1997, The Lancet.

[13]  G. Smith,et al.  Bias in meta-analysis detected by a simple, graphical test , 1997, BMJ.

[14]  N. Eastman.  Inquiry into homicides by psychiatric patients: systematic audit should replace mandatory inquiries , 1996, BMJ.

[15]  Lisa A. Weissfeld,et al.  An assessment of the use of the continuity correction for sparse data in meta-analysis , 1996 .

[16]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[17]  C. Begg,et al.  Operating characteristics of a rank correlation test for publication bias. , 1994, Biometrics.

[18]  P. Easterbrook.  Directory of registries of clinical trials. , 1992, Statistics in medicine.

[19]  P. Sleight,et al.  Publication bias , 1991, The Lancet.

[20]  N. Kaye,et al.  The psychiatrist's role, responses, and responsibilities when a patient commits suicide. , 1991, The American journal of psychiatry.

[21]  W. Menninger.  Patient suicide and its impact on the psychotherapist. , 1991, Bulletin of the Menninger Clinic.

[22]  G. Truant,et al.  Psychiatrists' experience of suicide in their patients. , 1990, Psychiatric journal of the University of Ottawa : Revue de psychiatrie de l'Universite d'Ottawa.

[23]  S. Bartels.  The aftermath of suicide on the psychiatric inpatient unit. , 1987, General hospital psychiatry.

[24]  L. S. Goldstein,et al.  Psychotherapists as suicide survivors. , 1984, American journal of psychotherapy.

[25]  P. Slawson,et al.  Suicide in the psychiatric hospital. , 1975, The American journal of psychiatry.

[26]  B. Barraclough,et al.  The aftermath of suicide. , 1974, British medical journal.