Outcome selection bias in meta-analysis

Publication bias has been previously identified as a threat to the validity of a meta-analysis. Recently, new evidence has documented an additional threat to validity, the selective reporting of trial outcomes within published studies. Several diseases have several possible measures of outcome. Some articles might report only a selection of those outcomes, perhaps those with statistically significant results. In this article, we review this problem while addressing the questions: what is within-study selective reporting? how common is it? why is it done? how can it mislead? how can it be detected?, and finally, what is the solution? We recommend that both publication bias and selective reporting should be routinely investigated in systematic reviews.

[1]  C. Gamble,et al.  Identification and impact of outcome selection bias in meta‐analysis , 2005, Statistics in medicine.

[2]  Ida Sim,et al.  Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1) , 2005, BMJ : British Medical Journal.

[3]  D. Altman,et al.  Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors , 2005, BMJ : British Medical Journal.

[4]  Carl V Phillips,et al.  Publication bias in situ , 2004 .

[5]  A. Hrõbjartsson,et al.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. , 2004, JAMA.

[6]  Deborah Ashby,et al.  Adjusting for publication bias: modelling the selection process. , 2004, Journal of evaluation in clinical practice.

[7]  A. Chan.  Outcome reporting bias in randomised trials , 2003 .

[8]  T. Albert,et al.  Bottom drawer papers: another waste of clinicians' time , 2002, Medical education.

[9]  S. Hahn,et al.  Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. , 2002, Journal of evaluation in clinical practice.

[10]  Douglas G. Altman,et al.  Systematic reviews in health care : meta-analysis in context , 2008 .

[11]  D J Hutchon,et al.  Publishing raw data and real time statistical analysis on e-journals , 2001, BMJ : British Medical Journal.

[12]  Douglas G. Altman,et al.  Systematic Reviews in Health Care , 2001 .

[13]  J L Hutton,et al.  Assessing the potential for bias in meta-analysis due to selective reporting of subgroup analyses within studies. , 2000, Statistics in medicine.

[14]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[15]  P. Williamson,et al.  Are systematic reviews taking heterogeneity into account? An analysis from the Infectious Diseases Module of the Cochrane Library. , 2000, Journal of evaluation in clinical practice.

[16]  R. Tweedie,et al.  A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis , 2000 .

[17]  C. Adams,et al.  Unpublished rating scales: A major source of bias in randomised controlled trials of treatments for schizophrenia , 2000, British Journal of Psychiatry.

[18]  P. Williamson,et al.  Bias in meta‐analysis due to outcome variable selection within studies , 2000 .

[19]  Douglas G Altman,et al.  How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing , 1999, The Lancet.

[20]  J. Copas.  What works?: selectivity models and meta‐analysis , 1999 .

[21]  C. Adams,et al.  Content and quality of 2000 controlled trials in schizophrenia over 50 years , 1998, BMJ.

[22]  A. Berger.  Cochrane Injuries Group Albumin ReviewersWhy albumin may not work , 1998 .

[23]  Cochrane Injuries,et al.  Human albumin administration in critically ill patients: systematic review of randomised controlled trials. , 1998, BMJ.

[24]  L. Bero,et al.  Publication bias and research on passive smoking: comparison of published and unpublished studies. , 1998, JAMA.

[25]  R. Wears,et al.  Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. , 1998, JAMA.

[26]  R. Simes,et al.  Publication bias: evidence of delayed publication in a cohort study of clinical research projects , 1997, BMJ.

[27]  K. Dickersin.  How important is publication bias? A synthesis of available data. , 1997, AIDS education and prevention : official publication of the International Society for AIDS Education.

[28]  I. Tannock,et al.  False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons. , 1996, Journal of the National Cancer Institute.

[29]  L. Stewart,et al.  Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. , 1995, Statistics in medicine.

[30]  Colin B. Begg,et al.  An Approach for Assessing Publication Bias Prior to Performing a Meta-Analysis , 1992 .

[31]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[32]  C. Begg,et al.  Publication bias : a problem in interpreting medical data , 1988 .

[33]  Joel B. Greenhouse,et al.  Selection Models and the File Drawer Problem , 1988 .

[34]  K. Dickersin,et al.  Publication bias and clinical trials. , 1987, Controlled clinical trials.

[35]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[36]  A. Greenwald.  Consequences of Prejudice Against the Null Hypothesis , 1975 .