Are studies reporting significant results more likely to be published?

INTRODUCTION Our objective was to assess the hypothesis that there are variations of the proportion of articles reporting a significant effect, with a higher percentage of those articles published in journals with impact factors. METHODS The contents of 5 orthodontic journals (American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, European Journal of Orthodontics, Journal of Orthodontics, and Orthodontics and Craniofacial Research), published between 2004 and 2008, were hand-searched. Articles with statistical analysis of data were included in the study and classified into 4 categories: behavior and psychology, biomaterials and biomechanics, diagnostic procedures and treatment, and craniofacial growth, morphology, and genetics. In total, 2622 articles were examined, with 1785 included in the analysis. Univariate and multivariate logistic regression analyses were applied with statistical significance as the dependent variable, and whether the journal had an impact factor, the subject, and the year were the independent predictors. RESULTS A higher percentage of articles showed significant results relative to those without significant associations (on average, 88% vs 12%) for those journals. Overall, these journals published significantly more studies with significant results, ranging from 75% to 90% (P = 0.02). Multivariate modeling showed that journals with impact factors had a 100% increased probability of publishing a statistically significant result compared with journals with no impact factor (odds ratio [OR], 1.99; 95% CI, 1.19-3.31). Compared with articles on biomaterials and biomechanics, all other subject categories showed lower probabilities of significant results. Nonsignificant findings in behavior and psychology and diagnosis and treatment were 1.8 (OR, 1.75; 95% CI, 1.51-2.67) and 3.5 (OR, 3.50; 95% CI, 2.27-5.37) times more likely to be published, respectively. CONCLUSIONS Journals seem to prefer reporting significant results; this might be because of authors' perceptions of the importance of their findings and editors' and reviewers' preferences for significant results. The implication of this factor in the reliability of systematic reviews is discussed.

[1]  H. Dubben,et al.  Systematic review of publication bias in studies on publication bias , 2005, BMJ : British Medical Journal.

[2]  K. Dickersin.  How important is publication bias? A synthesis of available data. , 1997, AIDS education and prevention : official publication of the International Society for AIDS Education.

[3]  George Davey Smith,et al.  meta-analysis bias in location and selection of studies , 1998 .

[4]  S. Shott,et al.  Association Between Funding Source and Study Outcome in Orthopaedic Research , 2003, Clinical orthopaedics and related research.

[5]  D. Altman,et al.  Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors , 2005, BMJ : British Medical Journal.

[6]  S. Ceci,et al.  Peer-review practices of psychological journals: The fate of published articles, submitted again , 1982, Behavioral and Brain Sciences.

[7]  R. Davidson.  Source of funding and outcome of clinical trials , 1986, Journal of general internal medicine.

[8]  George Davey Smith,et al.  Misleading meta-analysis , 1995, BMJ.

[9]  J. R. Gilbert,et al.  Is there gender bias in JAMA's peer review process? , 1994, JAMA.

[10]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[11]  A. Vickers,et al.  Do certain countries produce only positive results? A systematic review of controlled trials. , 1998, Controlled clinical trials.

[12]  C. Tudor-Smith,et al.  Healthy eating in Wales , 1995, BMJ.

[13]  Theodore Eliades,et al.  Impact Factor A Review with Specific Relevance to Orthodontic Journals , 2001, Journal of Orofacial Orthopedics-fortschritte Der Kieferorthopadie.

[14]  T. Eliades,et al.  Advanced orthodontic education: evolution of assessment criteria and methods to meet future challenges. , 2005, The Angle orthodontist.

[15]  J. Farndon,et al.  A comparison of reports from referees chosen by authors or journal editors in the peer review process. , 2000, Annals of the Royal College of Surgeons of England.

[16]  F. Wolf,et al.  Commercially funded and United States-based research is more likely to be published; good-quality studies with negative outcomes are not. , 2007, The Journal of bone and joint surgery. American volume.

[17]  Michael K. Landi,et al.  Positive-outcome bias: comparison of emergency medicine and general medicine literatures. , 2008, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[18]  Emil H Schemitsch,et al.  Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. , 2004, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[19]  Christian Gluud,et al.  Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? , 2003, JAMA.

[20]  Scientific publishing: The criteria used by editors of scientific dental journals in the assessment of manuscripts submitted for publication. , 1999, British Dental Journal.

[21]  P. Lee,et al.  Publication bias in meta-analysis: its causes and consequences. , 2000, Journal of clinical epidemiology.

[22]  K. Dickersin.  The existence of publication bias and risk factors for its occurrence. , 1990, JAMA.

[23]  K. Dickersin,et al.  Publication Bias: The Problem That Won't Go Away , 1993, Annals of the New York Academy of Sciences.

[24]  R. Simes,et al.  Publication bias: evidence of delayed publication in a cohort study of clinical research projects , 1997, BMJ.

[25]  T. Jefferson,et al.  Peer Review in Health Sciences , 1999 .

[26]  M Egger,et al.  Grey literature in meta-analyses of randomized trials of health care interventions. , 2007, The Cochrane database of systematic reviews.

[27]  B. Knishkowy,et al.  The association between funding by commercial interests and study outcome in randomized controlled drug trials. , 2001, Family practice.

[28]  Douglas G. Altman,et al.  Systematic Reviews in Health Care: Meta-Analysis in Context: Second Edition , 2008 .

[29]  J. Harrison,et al.  Delay and failure to publish dental research , 2005, Evidence-Based Dentistry.

[30]  L. Stewart,et al.  Time to publication for results of clinical trials. , 2007, The Cochrane database of systematic reviews.

[31]  I Chalmers,et al.  Underreporting research is scientific misconduct. , 1990, JAMA.

[32]  K. Dickersin,et al.  Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. , 1992, JAMA.

[33]  S. Brophy,et al.  Interventions for latent autoimmune diabetes (LADA) in adults. , 2011, The Cochrane database of systematic reviews.

[34]  Peter C Gøtzsche,et al.  Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review , 2006, BMJ : British Medical Journal.

[35]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .