Blind Peer Review by Academic Journals

The peer review process is central to the advancement of scholarly knowledge and is vital to the advancement of individual careers. With so much at stake, it is important to examine, and reexamine, issues pertaining to the fairness and quality of peer review on an ongoing basis. One central issue is blinding. In this chapter, we review evidence from across a variety of disciplines on the efficacy of blinding as a means of reducing bias and yielding higher quality reviews. We find that evidence is, in general, inconclusive. Most, though not all, benefits attributable to blinding are contingent on blinding being successful. Therefore, we also review evidence on the feasibility and efficacy of blinding and find that blinding is moderately effective. We conclude by examining the views of stakeholders in the peer review process and determine that, although the evidence is mixed, there is broad support for blinding.

[1]  G. Wilkinson,et al.  Open peer review: A randomised controlled trial , 2000, British Journal of Psychiatry.

[2]  Christian Genest,et al.  [Report of the Ad Hoc Committee on Design of an Experiment on Double-Blind Refereeing]: Comment , 1993 .

[3]  D. Laband,et al.  A citation analysis of the impact of blinded peer review. , 1994, JAMA.

[4]  Philip Goldberg,et al.  Are women prejudiced against women? , 1968 .

[5]  D. Horrobin.  Peer review: A philosophically faulty concept which is proving disastrous for science , 1982, Behavioral and Brain Sciences.

[6]  Peer review: time for a change? , 1988, Trends in pharmacological sciences.

[7]  Fiona Godlee,et al.  Effect of blinding and unmasking on the quality of peer review , 1999, Journal of General Internal Medicine.

[8]  P. Sahni,et al.  Does exchanging comments of Indian and non-Indian reviewers improve the quality of manuscript reviews? , 1999, The National medical journal of India.

[9]  D. Katz,et al.  Incidence and nature of unblinding by authors: our experience at two radiology journals with double-blinded peer review policies. , 2002, AJR. American journal of roentgenology.

[10]  Richard Smith,et al.  Peer Review: A Flawed Process at the Heart of Science and Journals , 2006 .

[11]  S. B. Friedman,et al.  The effects of blinding on acceptance of research papers by peer review. , 1994, JAMA.

[12]  C. Gross,et al.  Effect of blinded peer review on abstract acceptance. , 2006, JAMA.

[13]  R. Fletcher,et al.  The effects of blinding on the quality of peer review. A randomized trial. , 1990, JAMA.

[14]  Pamela P. Sawallis,et al.  Accuracy in the Identification of Scholarly and Peer-Reviewed Journals and the Peer-Review Process Across Disciplines , 2003 .

[15]  S. Isenberg,et al.  The effect of masking manuscripts for the peer-review process of an ophthalmic journal , 2009, British Journal of Ophthalmology.

[16]  Stephen J. Ceci,et al.  How blind is blind review , 1984 .

[17]  D. Rennie,et al.  Masking author identity in peer review: what factors influence masking success? PEER Investigators. , 1998, JAMA.

[18]  Robert P Freckleton,et al.  Does double-blind review benefit female authors? , 2008, Trends in ecology & evolution.

[19]  G. Saha.  Meeting the needs! , 2017, Indian journal of psychiatry.

[20]  Anthony K. H. Tung.  Impact of double blind reviewing on SIGMOD publication: a more detail analysis , 2006, SGMD.

[21]  D. Benos,et al.  The ups and downs of peer review. , 2007, Advances in physiology education.

[22]  James V. Bradley,et al.  Pernicious publication practices , 1981 .

[23]  M. Hicks,et al.  Evaluating peer reviews. Pilot testing of a grading instrument. , 1994, JAMA.

[24]  Jack Meadows,et al.  Editorial Peer Review: Its Strengths and Weaknesses , 2002, J. Documentation.

[25]  A R Jadad,et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary? , 1996, Controlled clinical trials.

[26]  A. Kazdin,et al.  Getting published , 2005, Cognitive Therapy and Research.

[27]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 2 , 1998 .

[28]  R. Merton,et al.  Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system , 1971 .

[29]  A. Caelleigh,et al.  A Tool for Reviewers: “Review Criteria for Research Manuscripts” , 2001 .

[30]  L. Billard.  [Report of the Ad Hoc Committee on Design of an Experiment on Double-Blind Refereeing]: Comment , 1993 .

[31]  Foster J. Provost,et al.  The myth of the double-blind review?: author identification using only citations , 2003, SKDD.

[32]  David B. Resnik,et al.  Perceptions of Ethical Problems with Scientific Journal Peer Review: An Exploratory Study , 2008, Sci. Eng. Ethics.

[33]  A. Link.  US and non-US submissions: an analysis of reviewer bias. , 1998, JAMA.

[34]  M. Kearney,et al.  Nurse editors' views on the peer review process. , 2005, Research in nursing & health.

[35]  A. Yankauer,et al.  How blind is blind review? , 1991, American journal of public health.

[36]  N. Black,et al.  Effect of blinding and unmasking on the quality of peer review: a randomized trial. , 1998, JAMA.

[37]  F. Godlee,et al.  Effect of open peer review on quality of reviews and on reviewers'recommendations: a randomised trial , 1999, BMJ.

[38]  Glenn Regehr,et al.  To blind or not to blind? What authors and reviewers prefer , 2006, Medical education.

[39]  E. Lawson,et al.  Effect of institutional prestige on reviewers' recommendations and editorial decisions. , 1994, JAMA.

[40]  T. Jefferson,et al.  Editorial peer review for improving the quality of reports of biomedical studies. , 2007, The Cochrane database of systematic reviews.

[41]  M. Hojat,et al.  Impartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review Process , 2003, Advances in health sciences education : theory and practice.

[42]  Margaret Stieg Dalton.  Refereeing of Scholarly Works for Primary Publishing. , 1995 .

[43]  T. Tregenza,et al.  Double-blind review favours increased representation of female authors. , 2008, Trends in ecology & evolution.

[44]  Molly C Dougherty,et al.  Blinding in peer review: the preferences of reviewers for nursing journals. , 2008, Journal of advanced nursing.

[45]  Richard T. Snodgrass,et al.  Single- versus double-blind reviewing: an analysis of the literature , 2006, SGMD.

[46]  Robert H. Fletcher,et al.  Evidence for the effectiveness of peer review , 1997 .

[47]  Cassidy R. Sugimoto,et al.  Bias in peer review , 2013, J. Assoc. Inf. Sci. Technol..

[48]  Art B. Owen,et al.  Meeting the Needs of New Statistical Researchers , 1991 .

[49]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 1 , 1998 .

[50]  D. Ratner,et al.  Blinded vs. unblinded peer review of manuscripts submitted to a dermatology journal: a randomized multi‐rater study , 2011, The British journal of dermatology.

[51]  Tony Delamothe,et al.  Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial , 2010, BMJ : British Medical Journal.

[52]  J. Scott.  Peer Review for Journals: Evidence on Quality Control, Fairness, and Innovation , 1997 .

[53]  R. Blank.  The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review , 1991 .

[54]  Håkan J. Holm,et al.  Double-blind in light of the internet: A note on author anonymity , 2011, Inf. Econ. Policy.

[55]  D. Rennie,et al.  Does masking author identity improve peer review quality? A randomized controlled trial. PEER Investigators. , 1998, JAMA.

[56]  Richard T. Snodgrass,et al.  Editorial: Single- versus double-blind reviewing , 2007, TODS.

[57]  S. Ceci,et al.  Peer-review practices of psychological journals: The fate of published articles, submitted again , 1982, Behavioral and Brain Sciences.

[58]  D. Cox,et al.  Report of the Ad Hoc Committee on Double-Blind Refereeing , 1993 .

[59]  Peter A. Abrams,et al.  The Predictive Ability of Peer Review of Grant Proposals: The Case of Ecology and the US National Science Foundation , 1991 .

[60]  F. Godlee,et al.  Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. , 1998, JAMA.

[61]  John Spencer,et al.  Reviewers' perceptions of the peer review process for a medical education journal , 2005, Medical education.

[62]  Stuart A. Kirk,et al.  Recognition of Authors in Blind Review of Manuscripts , 1981 .

[63]  Janet K. Swim,et al.  Joan McKay versus John McKay: Do gender stereotypes bias evaluations? , 1989 .

[64]  R M Pitkin.  Blinded manuscript review: an idea whose time has come? , 1995, Obstetrics and gynecology.

[65]  Robert Rosenthal,et al.  Reliability and bias in peer-review practices , 1982, Behavioral and Brain Sciences.

[66]  David J. DeWitt,et al.  Impact of double-blind reviewing on SIGMOD publication rates , 2006, SGMD.

[67]  R Smith,et al.  Opening up BMJ peer review , 1999, BMJ.

[68]  C. Wennerås,et al.  Nepotism and sexism in peer-review , 1997, Nature.

[69]  D. Perlman,et al.  Reviewer “bias”: Do Peters and Ceci protest too much? , 1982, Behavioral and Brain Sciences.