Peer Review: Recent Experience and Future Directions

What are the challenges and opportunities facing peer review in a networked world? We review the state of journal peer review at the end of 2010, beset by ever-increasing volumes of journal articles demanding to be reviewed in ever-shorter times and by a range of criticisms of its effectiveness and value. We consider the opportunities for improvement and review alternative approaches including open and post-publication review and the possible use of article-level metrics as a replacement. We conclude that far from being in crisis, peer review has remains widely supported and diversely innovative.

[1]  Fiona Godlee,et al.  Adequacy of authors’ replies to criticism raised in electronic letters to the editor: cohort study , 2010, BMJ : British Medical Journal.

[2]  Sara Schroter,et al.  Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. , 2006, JAMA.

[3]  Axel Boldt,et al.  Extending ArXiv.org to Achieve Open Peer Review and Publishing , 2010, ArXiv.

[4]  Robert P Crease,et al.  Your favourite units , 2010 .

[5]  Fabio Casati,et al.  Is peer review any good? A quantitative analysis of peer review , 2009 .

[6]  Mary Kalantzis,et al.  Signs of Epistemic Disruption: Transformations in the Knowledge System of the Academic Journal , 2009, First Monday.

[7]  Mahmod Noor Mazlina.  The University Of Queensland School Of Information Technology And Electrical Engineering , 2007 .

[8]  Jennifer Ann Lean.  Web , 2006 .

[9]  Stefan Thurner,et al.  Peer-review in a world with rational scientists: Toward selection of the average , 2010, 1008.4324.

[10]  Mark Ware,et al.  Peer review in scholarly journals: Perspective of the scholarly community - Results from an international study , 2008, Inf. Serv. Use.

[11]  Philip E. Bourne,et al.  What Do I Want from the Publisher of the Future? , 2010, PLoS Comput. Biol..

[12]  Eugene V. Koonin,et al.  Reviving a culture of scientific debate , 2006 .

[13]  C. Jennings.  Quality and value: The true purpose of peer review , 2006 .

[14]  L. Bornmann,et al.  A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants , 2010, PloS one.

[15]  Trish Groves.  How can we get the best out of peer review , 2006 .

[16]  Lutz Bornmann,et al.  The luck of the referee draw: the effect of exchanging reviews , 2009, Learn. Publ..

[17]  M. Andrade,et al.  Increasing Accountability , 2010 .

[18]  P. Lawrence.  The mismeasurement of science , 2007, Current Biology.

[19]  P. Rothwell,et al.  Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? , 2000, Brain : a journal of neurology.

[20]  Rob Procter,et al.  If you build it, will they come? How researchers perceive and use web 2.0 , 2010 .

[21]  Owen L. Petchey,et al.  Pubcreds: Fixing the Peer Review Process by “Privatizing” the Reviewer Commons , 2010 .

[22]  Karim Khan.  Is open peer review the fairest system? No , 2010, BMJ : British Medical Journal.

[23]  Ulrich Pöschl,et al.  An open, two-stage peer-review journal , 2006 .

[24]  Theodora Bloom.  Online frontiers of the peer-reviewed literature , 2006 .

[25]  Raymond S. Nickerson,et al.  What authors want from journal reviewers and editors , 2005 .

[26]  T. Groves.  Is open peer review the fairest system? Yes , 2010, BMJ : British Medical Journal.

[27]  Lutz Bornmann,et al.  Do women have less success in peer review , 2009 .

[28]  Chris Armbruster.  Moving out of Oldenburg's Long Shadow: What is the Future for Society Publishing? , 2007 .

[29]  Tony Delamothe,et al.  Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial , 2010, BMJ : British Medical Journal.

[30]  J. Maunsell.  Announcement Regarding Supplemental Material , 2010 .

[31]  H. Marsh,et al.  Improving the Peer-review Process for Grant Applications , 2022 .

[32]  Mark Ware.  Current Peer Review Practice and Perceptions: The View from the Field , 2013 .

[33]  Suzie Allard,et al.  Research Publication Characteristics and Their Relative Values: A , 2010 .

[34]  J. Ioannidis,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias , 2008, PloS one.

[35]  T. Jefferson,et al.  Editorial peer review for improving the quality of reports of biomedical studies. , 2007, The Cochrane database of systematic reviews.

[36]  Elizabeth Wager,et al.  What is it for? Analysing the purpose of peer review. , 2006 .

[37]  Rob Procter,et al.  If you build it, will they come? : how researchers perceive and use web 2.0 : a Research Information Network report , 2010 .

[38]  Bernd Pulverer,et al.  Transparency showcases strength of peer review , 2010, Nature.

[39]  Lutz Bornmann,et al.  Do Author-Suggested Reviewers Rate Submissions More Favorably than Editor-Suggested Reviewers? A Study on Atmospheric Chemistry and Physics , 2010, PloS one.

[40]  D. Rennie,et al.  Congress on Biomedical Peer Review: history, ethics, and plans for the future. , 1998, JAMA.

[41]  L. Rieseberg,et al.  No crisis in supply of peer reviewers , 2010, Nature.

[42]  Bo-Christer Björk,et al.  Scientific journal publishing: yearly volume and open access availability , 2009, Inf. Res..

[43]  F. Godlee,et al.  Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. , 1998, JAMA.

[44]  Lutz Bornmann,et al.  Reliability of reviewers' ratings when using public peer review: a case study , 2010, Learn. Publ..

[45]  Richard Smith,et al.  Peer Review: A Flawed Process at the Heart of Science and Journals , 2006, Journal of the Royal Society of Medicine.

[46]  Peter Frishauf,et al.  Reputation Systems: A New Vision for Publishing and Peer Review , 2009 .

[47]  Richard W. Smith.  In Search Of an Optimal Peer Review System , 2009 .

[48]  Ulrich Pöschl,et al.  Interactive Open Access Publishing and Peer Review: The Effectiveness and Perspectives of Transparency and Self-Regulation in Scientific Communication and Evaluation , 2010 .