Generalized synthesis of evidence and the threat of dissemination bias. the example of electronic fetal heart rate monitoring (EFM).

Assessment of the potential impact of dissemination bias is necessary for meta-analysis. When evidence is available from studies of different designs, the different study types may be affected by dissemination bias to differing degrees. The evidence relating to electronic fetal heart rate monitoring (EFM) for preventing perinatal mortality is used to explore the feasibility of carrying out a dissemination bias assessment in a generalized synthesis of evidence (gse) framework. Visual inspection of funnel plots, statistical tests, and methods to "adjust" the results of a meta-analysis are all used in an extensive sensitivity analysis. The potential impact of dissemination bias on gse models synthesizing all the evidence together is also reported. Detailed consideration is given to the influence of meta-analysis model choice, and outcome scale used. Using the risk difference scale, funnel plots of the observational studies appeared highly asymmetric. However, further explorations show these conclusions are not robust over use of different outcome measures or different meta-analysis models. Researchers should be aware that dissemination bias may affect different sources of evidence differently. Although assessments such as those described here are recommended, awareness of their lack of robustness to outcome scale and model choice is important. Further research into methods to assess dissemination bias that are invariant to these factors is needed.

[1]  J. L. Tang,et al.  Misleading funnel plot for detection of bias in meta-analysis. , 2000, Journal of clinical epidemiology.

[2]  Douglas G. Altman,et al.  Systematic Reviews in Health Care: Meta-Analysis in Context: Second Edition , 2008 .

[3]  R. Simes Publication bias: the case for an international registry of clinical trials. , 1986, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[4]  Lisa A. Weissfeld,et al.  An assessment of the use of the continuity correction for sparse data in meta-analysis , 1996 .

[5]  S G Thompson,et al.  Analysing the relationship between treatment effect and underlying risk in meta-analysis: comparison and development of approaches. , 2000, Statistics in medicine.

[6]  A. Hartz,et al.  A comparison of observational studies and randomized, controlled trials , 2000, American journal of ophthalmology.

[7]  J. Copas,et al.  Reanalysis of epidemiological evidence on lung cancer and passive smoking , 2000, BMJ : British Medical Journal.

[8]  K R Abrams,et al.  Bayesian methods in meta-analysis and evidence synthesis. , 2001, Statistical methods in medical research.

[9]  J. Ioannidis Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. , 1998, JAMA.

[10]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[11]  David R. Jones,et al.  Hierarchical models in generalized synthesis of evidence: an example based on studies of breast cancer screening. , 2000, Statistics in medicine.

[12]  S D Walter,et al.  A comparison of methods to detect publication bias in meta‐analysis , 2001, Statistics in medicine.

[13]  I Olkin,et al.  Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. , 2000, Statistics in medicine.

[14]  A D Oxman,et al.  The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials , 1998, BMJ.

[15]  J. L. Tang,et al.  Weighting bias in meta-analysis of binary outcomes. , 2000, Journal of clinical epidemiology.

[16]  T. Sheldon,et al.  Relation between hostility and coronary heart disease , 1999, British medical journal.

[17]  G. Smith,et al.  Bias in meta-analysis detected by a simple, graphical test , 1997, BMJ.

[18]  C Bain,et al.  Interpreting the evidence: choosing between randomised and non-randomised studies , 1999, BMJ.

[19]  Alex J Sutton,et al.  Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. , 2002, International journal of epidemiology.

[20]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[21]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[22]  R Platt,et al.  Is There a “Best” Way to Detect and Minimize Publication Bias? , 2001, Evaluation & the health professions.

[23]  R. Smith What is publication? , 1999, BMJ.

[24]  Des Spence Interpreting the evidence , 2002, BMJ : British Medical Journal.

[25]  N. Black Why we need observational studies to evaluate the effectiveness of health care , 1996, BMJ.

[26]  D. Spiegelhalter,et al.  Bayesian random effects meta‐analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales , 2002, Statistics in medicine.

[27]  P. Lee,et al.  Publication bias in meta-analysis: its causes and consequences. , 2000, Journal of clinical epidemiology.

[28]  J. Concato,et al.  Randomized, controlled trials, observational studies, and the hierarchy of research designs. , 2000, The New England journal of medicine.

[29]  M. Egger,et al.  The hazards of scoring the quality of clinical trials for meta-analysis. , 1999, JAMA.

[30]  J G Thornton,et al.  Bayesian interpretation of trials: the example of intrapartum electronic fetal heart rate monitoring , 2000, BJOG : an international journal of obstetrics and gynaecology.

[31]  Jonathan A C Sterne,et al.  Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. , 2001, BMJ.

[32]  David R. Jones,et al.  Empirical assessment of effect of publication bias on meta-analyses , 2000, BMJ : British Medical Journal.

[33]  Alex J. Sutton,et al.  Modelling publication bias in meta-analysis: a review , 2000 .

[34]  R. Tweedie,et al.  Publication Bias in Meta-Analysis: A Bayesian Data-Augmentation Approach to Account for Issues Exempli(cid:12)ed in the Passive Smoking Debate , 1997 .

[35]  R. Simes,et al.  Publication bias: evidence of delayed publication in a cohort study of clinical research projects , 1997, BMJ.

[36]  R. Tweedie,et al.  A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis , 2000 .

[37]  J. Ioannidis,et al.  Comparison of evidence of treatment effects in randomized and nonrandomized studies. , 2001, JAMA.

[38]  Alex J. Sutton,et al.  Publication and related biases: a review , 2000 .

[39]  S D Walter,et al.  Choice of effect measure for epidemiological data. , 2000, Journal of clinical epidemiology.

[40]  Douglas G. Altman,et al.  Systematic Reviews in Health Care , 2001 .