Scientific Utopia: I. Opening Scientific Communication

Existing norms for scientific communication are rooted in anachronistic practices of bygone eras making them needlessly inefficient. We outline a path that moves away from the existing model of scientific communication to improve the efficiency in meeting the purpose of public science—knowledge accumulation. We call for six changes: (a) full embrace of digital communication; (b) open access to all published research; (c) disentangling publication from evaluation; (d) breaking the “one article, one journal” model with a grading system for evaluation and diversified dissemination outlets; (e) publishing peer review; and (f) allowing open, continuous peer review. We address conceptual and practical barriers to change and provide examples showing how the suggested practices are being used already. The critical barriers to change are not technical or financial; they are social. Although scientists guard the status quo, they also have the power to change it.

[1]  Arif E. Jinha.  Article 50 million: an estimate of the number of scholarly articles in existence , 2010, Learn. Publ..

[2]  A. Greenwald,et al.  Measuring individual differences in implicit cognition: the implicit association test. , 1998, Journal of personality and social psychology.

[3]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[4]  Wolfgang Viechtbauer,et al.  Publication bias in meta-analysis: Prevention, assessment and adjustments , 2007, Psychometrika.

[5]  Steven J. Bell,et al.  The blended librarian: A blueprint for redefining the teaching and learning role of academic librarians , 2004 .

[6]  A. Greenwald.  Consequences of Prejudice Against the Null Hypothesis , 1975 .

[7]  David R. Jones,et al.  Empirical assessment of effect of publication bias on meta-analyses , 2000, BMJ : British Medical Journal.

[8]  Leandre R. Fabrigar,et al.  The Review Process at PSPB: Correlates of Interreviewer Agreement and Manuscript Acceptance , 1999 .

[9]  Jelte M. Wicherts,et al.  Psychology must learn a lesson from fraud case , 2011, Nature.

[10]  David Berle,et al.  Inconsistencies between reported test statistics and p‐values in two psychiatry journals , 2007, International journal of methods in psychiatric research.

[11]  F. Godlee,et al.  Effect of open peer review on quality of reviews and on reviewers'recommendations: a randomised trial , 1999, BMJ.

[12]  Daniele Fanelli,et al.  Negative results are disappearing from most disciplines and countries , 2011, Scientometrics.

[13]  S Goldbeck-Wood,et al.  Evidence on peer review—scientific quality control or smokescreen? , 1999, BMJ.

[14]  J. Glynn,et al.  Open access policy , 2013, The Lancet.

[15]  Brian D. Cameron,et al.  Trends in the Usage of ISI Bibliometric Data: Uses, Abuses, and Implications , 2005 .

[16]  Stevan Harnad,et al.  The invisible hand of peer review , 1998 .

[17]  Vincent Larivière,et al.  Self-Selected or Mandated, Open Access Increases Citation Impact for Higher Quality Research , 2010, PloS one.

[18]  J. Schooler.  Unpublished results hide the decline effect , 2011, Nature.

[19]  Samuel Ball,et al.  The Peer Review Process Used to Evaluate Manuscripts Submitted to Academic Journals: Interjudgmental Reliability , 1989 .

[20]  D. V. Essen,et al.  Cognitive neuroscience 2.0: building a cumulative science of human brain function , 2010, Trends in Cognitive Sciences.

[21]  Denny Borsboom,et al.  Letting the daylight in: Reviewing the reviewers and other ways to maximize transparency in science , 2012, Front. Comput. Neurosci..

[22]  David Green,et al.  An Open Access Overview , 2012 .

[23]  Charles Oppenheim,et al.  The citation advantage of open-access articles , 2008, J. Assoc. Inf. Sci. Technol..

[24]  Bo-Christer Björk,et al.  Scientific journal publishing: yearly volume and open access availability , 2009, Inf. Res..

[25]  Pamela W. Smith.  The National Institutes of Health (NIH) , 2008 .

[26]  M Sievert,et al.  Phenomena of retraction: reasons for retraction and citations to the publications. , 1998, JAMA.

[27]  M. Borenstein,et al.  Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments , 2006 .

[28]  William H. Starbuck,et al.  How Much Better are the Most Prestigious Journals? The Statistics of Academic Publication , 2005, Organ. Sci..

[29]  G. Smith,et al.  Bias in meta-analysis detected by a simple, graphical test , 1997, BMJ.

[30]  P. Tetlock,et al.  Accounting for the effects of accountability. , 1999, Psychological bulletin.

[31]  D. Fanelli.  “Positive” Results Increase Down the Hierarchy of the Sciences , 2010, PloS one.

[32]  Steven L. Miller,et al.  Neural deficits in children with dyslexia ameliorated by behavioral remediation: Evidence from functional MRI , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[33]  J. Ioannidis.  Why Most Published Research Findings Are False , 2005, PLoS medicine.

[34]  L. Bornmann,et al.  A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants , 2010, PloS one.

[35]  D. W. Fiske,et al.  But the Reviewers Are Making Different Criticisms of My Paper! Diversity and Uniqueness in Reviewer Comments. , 1990 .

[36]  Jim Giles,et al.  PR's 'pit bull' takes on open access , 2007, Nature.

[37]  C. Sunstein.  Republic.com 2.0 , 2007 .

[38]  J. Armstrong,et al.  Peer review for journals: Evidence on quality control, fairness, and innovation , 1997 .

[39]  Robert Rosenthal,et al.  How often are our numbers wrong , 1978 .

[40]  D F Horrobin,et al.  The philosophical basis of peer review and the suppression of innovation. , 1990, JAMA.

[41]  A. Greenwald,et al.  Under what conditions does theory obstruct research progress? , 1986, Psychological review.

[42]  R Smith,et al.  Opening up BMJ peer review , 1999, BMJ.

[43]  D. Benos,et al.  The ups and downs of peer review. , 2007, Advances in physiology education.

[44]  Joseph S. Rossi,et al.  How Often are Our Statistics Wrong? a Statistics Class Exercise , 1987 .

[45]  G. Eysenbach.  The Open Access Advantage , 2006, Journal of medical Internet research.

[46]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[47]  Burt V. Bronk,et al.  Hierarchy of sciences , 1977 .

[48]  S. Harnad,et al.  Open access to peer-reviewed research through author/institution self-archiving: maximizing research impact by maximizing online access. , 2003, Journal of postgraduate medicine.

[49]  Michael Jubb,et al.  Open Access - What Are the Economic Benefits? A Comparison of the United Kingdom, Netherlands and Denmark , 2010, Learn. Publ..

[50]  Per Ottar Seglen,et al.  Causal relationship between article citedness and journal impact , 1994 .

[51]  J. Ioannidis.  Why Most Published Research Findings Are False , 2005 .

[52]  Timothy D. Wilson,et al.  Scientists' Evaluations of Research: the Biasing Effects of the Importance of the Topic , 1993 .

[53]  Jindřiška Svobodová,et al.  How Science Works , 2014 .

[54]  E. Wagenmakers,et al.  Erroneous analyses of interactions in neuroscience: a problem of significance , 2011, Nature Neuroscience.

[55]  E. García‐Berthou,et al.  Incongruence between test statistics and P values in medical papers , 2004 .

[56]  Grover J. Whitehurst,et al.  Interrater agreement for journal manuscript reviews. , 1984 .

[57]  Brian A. Nosek,et al.  Implicit social cognition: from measures to mechanisms , 2011, Trends in Cognitive Sciences.

[58]  Philip E. Tetlock,et al.  Biases in research evaluation: Inflated assessment, oversight, or error-type weighting? , 2007 .

[59]  Patrick Onghena,et al.  An Assessment of the Predictive Validity of Impact Factor Scores: Implications for Academic Employment Decisions in Social Work , 2006 .

[60]  J. Houghton,et al.  Economic implications of alternative scholarly publishing models : exploring the costs and benefits. JISC EI-ASPM Project. A report to the Joint Information Systems Committee (JISC) , 2009 .

[61]  H. Yarandi,et al.  Empirical developments in retraction , 2008, Journal of Medical Ethics.

[62]  T. Sterling.  Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa , 1959 .

[63]  J. Wicherts,et al.  The (mis)reporting of statistical results in psychology journals , 2011, Behavior research methods.

[64]  Emma Marris.  American Chemical Society: Chemical reaction , 2005, Nature.

[65]  William Y. Arms.  What are the alternatives to peer review? Quality Control in Scholarly Publishing on the Web , 2002 .

[66]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[67]  J. Scott.  Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation , 1997 .

[68]  V. Stodden.  Trust your science? Open your data and code , 2011 .

[69]  David Moher,et al.  Comparison of registered and published primary outcomes in randomized controlled trials. , 2009, JAMA.

[70]  David J. Hardisty,et al.  Diffusion of treatment research: does Open Access matter? , 2008, Journal of clinical psychology.

[71]  N. Adler,et al.  When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings , 2009 .

[72]  Brian A. Nosek,et al.  Scientific Utopia , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[73]  John P A Ioannidis,et al.  Excess significance bias in the literature on brain volume abnormalities. , 2011, Archives of general psychiatry.

[74]  R. Merton,et al.  Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system , 1971 .

[75]  Fred Lewis Muhl.  A public library , 1904 .

[76]  Jennifer M. Urban,et al.  Shining Light into Black Boxes , 2012, Science.

[77]  John D. Norton,et al.  How science works , 1998 .

[78]  Matthew B. Jones,et al.  Challenges and Opportunities of Open Data in Ecology , 2011, Science.

[79]  Philip M. Davis,et al.  Open access publishing, article downloads, and citations: randomised controlled trial , 2008, BMJ : British Medical Journal.

[80]  D. Cicchetti.  The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation , 1991, Behavioral and Brain Sciences.

[81]  D. Borsboom,et al.  The poor availability of psychological research data for reanalysis. , 2006, The American psychologist.

[82]  Lowell L. Hargens,et al.  Scholarly Consensus and Journal Rejection Rates. , 1988 .

[83]  S. Ceci,et al.  Peer-review practices of psychological journals: The fate of published articles, submitted again , 1982, Behavioral and Brain Sciences.

[84]  P. Sleight,et al.  Publication bias , 1991, The Lancet.

[85]  Theodor D. Sterling,et al.  Publication decisions revisited: the effect of the outcome of statistical tests on the decision to p , 1995 .

[86]  K. Antelman.  Do Open-Access Articles Have a Greater Research Impact? , 2004 .