The Peer Reviewers' Openness Initiative: incentivizing open research practices through peer review

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle.

[1]  S. Ceci.  Scientists’ Attitudes toward Data Sharing , 1988 .

[2]  Spencer D. C. Keralis,et al.  The Problem of Data , 2012 .

[3]  W. Vanpaemel,et al.  Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm , 2015 .

[4]  Jeffrey N Rouder,et al.  The what, why, and how of born-open data , 2015, Behavior Research Methods.

[5]  Leroy Wolins,et al.  Responsibility for Raw Data. , 1962 .

[6]  Veerle Van den Eynden,et al.  Managing and Sharing Data , 2014 .

[7]  C. Menke.  A Note on Science and Democracy? Robert K. Mertons Ethos of Science , 2015 .

[8]  D. Borsboom,et al.  The poor availability of psychological research data for reanalysis. , 2006, The American psychologist.

[9]  Melissa S. Anderson,et al.  Data Withholding and the Next Generation of Scientists: Results of a National Survey , 2006, Academic medicine : journal of the Association of American Medical Colleges.

[10]  A. Vickers,et al.  Empirical Study of Data Sharing by Authors Publishing in PLoS Journals , 2009, PloS one.

[11]  No impact without data access , 2015, Nature Genetics.

[12]  Veerle Van den Eynden,et al.  Managing and sharing data: Best practice for researchers , 2011 .

[13]  S. Eddy,et al.  Sharing Publication-Related Data and Materials: Responsibilities of Authorship in the Life Sciences1 , 2003, Plant Physiology.

[14]  Geoffrey W. McCarthy,et al.  On Being a Scientist: A Guide to Responsible Conduct in Research, 3rd ed. , 2012 .

[15]  J. Wicherts.  Science revolves around the data , 2013 .

[16]  Friedrich Leisch,et al.  Sweave: Dynamic Generation of Statistical Reports Using Literate Data Analysis , 2002, COMPSTAT.

[17]  Division on Earth.  Sharing Publication-Related Data and Materials:: Responsibilities of Authorship in the Life Sciences , 2003 .

[18]  Brian A. Nosek,et al.  Scientific Utopia , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[19]  Matthew Cooper,et al.  Sharing Data and Results in Ethnographic Research: Why This Should Not Be an Ethical Imperative , 2007, Journal of empirical research on human research ethics : JERHRE.

[20]  A. Coughlan,et al.  An Empirical Investigation * , 2002 .

[21]  Brian A. Nosek,et al.  Promoting an open research culture , 2015, Science.

[22]  Marcia McNutt,et al.  Data sharing , 2016, Science.

[23]  Maria Victoria Schneider,et al.  The open science peer review oath. , 2014, F1000Research.

[24]  Carly Strasser,et al.  Primer on Data Management: What you always wanted to know , 2012 .

[25]  Nicholas R Cozzarelli,et al.  UPSIDE: Uniform principle for sharing integral data and materials expeditiously. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[26]  Similar Federal Agency Programs.  COMMITTEE ON SCIENCE, ENGINEERING, AND PUBLIC POLICY , 2013 .

[27]  Melissa S. Anderson,et al.  Normative Dissonance in Science: Results from a National Survey of U.S. Scientists , 2007, Journal of empirical research on human research ethics : JERHRE.

[28]  P. Allotey,et al.  Data sharing in medical research: an empirical investigation. , 2001, Bioethics.

[29]  F. Dominici,et al.  Reproducible epidemiologic research. , 2006, American journal of epidemiology.

[30]  K. A. Holmes,et al.  Results of a National Survey , 2016 .

[31]  Heather A. Piwowar,et al.  Sharing Detailed Research Data Is Associated with Increased Citation Rate , 2007, PloS one.

[32]  Matthew Gentzkow,et al.  Code and Data for the Social Sciences: A Practitioner's Guide , 2014 .

[33]  James R. Craig,et al.  Retention of raw data: A problem revisited. , 1973 .

[34]  John L. Florell,et al.  Sharing , 1979 .