How to Make More Published Research True

In a 2005 paper that has been accessed more than a million times, John Ioannidis explained why most published research findings were false. Here he revisits the topic, this time to address how to improve matters. Please see later in the article for the Editors' Summary

[1]  J. Ioannidis.  Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[2]  Nicholas C. Ide,et al.  Issues in the registration of clinical trials. , 2007, JAMA.

[3]  Iain Chalmers,et al.  How to increase value and reduce waste when research priorities are set , 2014, The Lancet.

[4]  D. Moher,et al.  A catalogue of reporting guidelines for health research , 2010, European journal of clinical investigation.

[5]  S. Stanley Young,et al.  Deming, data and observational studies , 2011 .

[6]  C. Begley,et al.  Drug development: Raise standards for preclinical cancer research , 2012, Nature.

[7]  John P A Ioannidis,et al.  The power of meta-analysis in genome-wide association studies. , 2013, Annual review of genomics and human genetics.

[8]  D. Rennie,et al.  Research on peer review and biomedical publication: furthering the quest to improve the quality of reporting. , 2014, JAMA.

[9]  F. Prinz,et al.  Believe it or not: how much can we rely on published data on potential drug targets? , 2011, Nature Reviews Drug Discovery.

[10]  C. Gamble,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review , 2013, PloS one.

[11]  J. Ioannidis,et al.  Research grants: Conform and be funded , 2012, Nature.

[12]  Robert Tibshirani,et al.  Scientific research in the age of omics: the good, the bad, and the sloppy , 2013, J. Am. Medical Informatics Assoc..

[13]  Kevin W Boyack,et al.  A list of highly influential biomedical researchers, 1996–2011 , 2013, European journal of clinical investigation.

[14]  J. Bohannon.  Who's afraid of peer review? , 2013, Science.

[15]  Brian A. Nosek,et al.  Scientific Utopia: I. Opening Scientific Communication , 2012, ArXiv.

[16]  Clifford R. Mynatt,et al.  Confirmation Bias in a Simulated Research Environment: An Experimental Study of Scientific Inference , 1977 .

[17]  F. Collins,et al.  NIH plans to enhance reproducibility , 2014 .

[18]  S. Goodman,et al.  Raw data from clinical trials: within reach? , 2013, Trends in pharmacological sciences.

[19]  Jessica D. Ritchie,et al.  Sea change in open science and data sharing: leadership by industry. , 2014, Circulation. Cardiovascular quality and outcomes.

[20]  S. Greenfield,et al.  Clinical practice guidelines we can trust , 2011 .

[21]  Tina Hernandez-Boussard,et al.  Overlapping meta-analyses on the same topic: survey of published studies , 2013, BMJ.

[22]  M. Rozing,et al.  Profit (p)-Index: The Degree to Which Authors Profit from Co-Authors , 2013, PloS one.

[23]  V. Johnson.  Revised standards for statistical evidence , 2013, Proceedings of the National Academy of Sciences.

[24]  John P. A. Ioannidis,et al.  More time for research: Fund people not projects , 2011, Nature.

[25]  George Poste,et al.  Biospecimens, biomarkers, and burgeoning data: the imperative for more rigorous research standards. , 2012, Trends in molecular medicine.

[26]  Jeffrey R. Spies,et al.  Scientific Utopia: II. Restructuring incentives and practices to promote truth over publishability , 2012, 1205.4251.

[27]  John P A Ioannidis,et al.  Life Cycle of Translational Research for Medical Interventions , 2008, Science.

[28]  Erika Check Hayden.  Cancer-gene data sharing boosted , 2014, Nature.

[29]  Alan F. Karr,et al.  Deming, data and observational studies: A process out of control and needing fixing , 2013 .

[30]  John P A Ioannidis,et al.  Is there a glass ceiling for highly cited scientists at the top of research universities? , 2010, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[31]  Richard Van Noorden.  China tops Europe in R&D intensity , 2014, Nature.

[32]  H. Beek.  F1000Prime recommendation of False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. , 2012 .

[33]  R. Nickerson.  Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[34]  John P. A. Ioannidis,et al.  Assessing value in biomedical research: the PQRST of appraisal and reward. , 2014, JAMA.

[35]  Birte U. Forstmann,et al.  Rewarding high-power replication research , 2014, Cortex.

[36]  Regina Nuzzo,et al.  Scientific method: Statistical errors , 2014, Nature.

[37]  Dimitri A Christakis,et al.  Rethinking reanalysis. , 2013, JAMA.

[38]  David L Donoho,et al.  An invitation to reproducible computational research. , 2010, Biostatistics.

[39]  Data sharing will pay dividends. , 2014, Nature.

[40]  Brian A. Nosek,et al.  Scientific Utopia , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[41]  F. Collins,et al.  Policy: NIH plans to enhance reproducibility , 2014, Nature.

[42]  H. Pashler,et al.  Is the Replicability Crisis Overblown? Three Arguments Examined , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[43]  Eduardo L. Franco,et al.  Making Prospective Registration of Observational Research a Reality , 2014, Science Translational Medicine.

[44]  M. Schein,et al.  Redundant surgical publications: tip of the iceberg? , 2001, Surgery.

[45]  M. Lauer,et al.  Percentile Ranking and Citation Impact of a Large Cohort of National Heart, Lung, and Blood Institute–Funded Cardiovascular R01 Grants , 2014, Circulation research.

[46]  J P Kassirer,et al.  The journal's policy on cost-effectiveness analyses. , 1994, The New England journal of medicine.

[47]  J. Ioannidis.  Why Most Published Research Findings Are False , 2005, PLoS medicine.

[48]  Gary S Collins,et al.  Impact of peer review on reports of randomised trials published in open peer review journals: retrospective before and after study , 2014, BMJ : British Medical Journal.

[49]  D. Rennie,et al.  Disclosure of Researcher Contributions: A Study of Original Research Articles in The Lancet , 1999, Annals of Internal Medicine.

[50]  Published Online.  Biomedical research: increasing value, reducing waste , 2014 .

[51]  Rustam Al-Shahi Salman,et al.  Increasing value and reducing waste in biomedical research regulation and management , 2014, The Lancet.

[52]  C. Monforton.  Weight of the evidence or wait for the evidence? Protecting underground miners from diesel particulate matter. , 2006, American journal of public health.

[53]  Muin J. Khoury,et al.  Knowledge integration at the center of genomic medicine , 2012, Genetics in Medicine.

[54]  Peter C Gøtzsche,et al.  Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review , 2006, BMJ : British Medical Journal.

[55]  D. Fanelli.  “Positive” Results Increase Down the Hierarchy of the Sciences , 2010, PloS one.

[56]  P. Donnelly,et al.  Replicating genotype–phenotype associations , 2007, Nature.

[57]  J. Ioannidis,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias , 2008, PloS one.

[58]  John P A Ioannidis,et al.  Content area experts as authors: helpful or harmful for systematic reviews and meta-analyses? , 2012, BMJ : British Medical Journal.

[59]  S. Lazic,et al.  A call for transparent reporting to optimize the predictive value of preclinical research , 2012, Nature.

[60]  Ana Marusic,et al.  Clinical trial registration: looking back and moving ahead , 2007, The Lancet.

[61]  T. Greenhalgh,et al.  Evidence based medicine: a movement in crisis? , 2014, BMJ : British Medical Journal.

[62]  Mina Bissell,et al.  Reproducibility: The risks of the replication drive , 2013, Nature.

[63]  C. Wennerås,et al.  Nepotism and sexism in peer-review , 1997, Nature.

[64]  F. Dominici,et al.  Reproducible epidemiologic research. , 2006, American journal of epidemiology.

[65]  Harlan M Krumholz,et al.  Increasing value and reducing waste: addressing inaccessible research , 2014, The Lancet.

[66]  J. Ross,et al.  Study of neurontin: titrate to effect, profile of safety (STEPS) trial: a narrative account of a gabapentin seeding trial. , 2011, Archives of internal medicine.

[67]  David Moher,et al.  Reducing waste from incomplete or unusable reports of biomedical research , 2014, The Lancet.

[68]  Malcolm Macleod,et al.  Why animal research needs to improve , 2011, Nature.

[69]  Nicholas C. Ide,et al.  The ClinicalTrials.gov results database--update and key issues. , 2011, The New England journal of medicine.

[70]  John P. A. Ioannidis,et al.  Undue industry influences that distort healthcare research, strategy, expenditure and practice: a review , 2013, European journal of clinical investigation.

[71]  R. Peng.  Reproducible Research in Computational Science , 2011, Science.

[72]  V. Stodden,et al.  Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals , 2013, PloS one.

[73]  J. Ioannidis,et al.  Why Current Publication Practices May Distort Science , 2008, PLoS medicine.

[74]  J. Ioannidis,et al.  Transforming Epidemiology for 21st Century Medicine and Public Health , 2013, Cancer Epidemiology, Biomarkers & Prevention.

[75]  J. Ioannidis,et al.  What's to Know About the Credibility of Empirical Economics? , 2013 .

[76]  Nils T. Hagen,et al.  Harmonic Allocation of Authorship Credit: Source-Level Correction of Bibliometric Bias Assures Accurate Publication and Citation Analysis , 2008, PloS one.

[77]  J. Ioannidis,et al.  The False-positive to False-negative Ratio in Epidemiologic Studies , 2011, Epidemiology.