Bias and effort in peer review

Here, we develop a theory of the relationship between the reviewer's effort and bias in peer review. From this theory, it follows that journal editors might employ biased reviewers because they shirk less. This creates an incentive for the editor to use monitoring mechanisms (e.g., associate editors supervising the peer review process) that mitigate the resulting bias in the reviewers' recommendations. The supervision of associate editors could encourage journal editors to employ more extreme reviewers. This theory helps to explain the presence of bias in peer review. To mitigate shirking by a reviewer, the journal editor may assign biased referees to generate information about the manuscript's quality and subject the reviewer's recommendations to supervision by a more aligned associate editor.

[1]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 2 , 1998 .

[2]  R. Nickerson Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[3]  Lutz Bornmann,et al.  The manuscript reviewing process: Empirical research on review requests, review sequences, and decision rules in peer review , 2010 .

[4]  Harold Maurice Collins,et al.  New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System , 1991 .

[5]  D. Chubin,et al.  Peerless Science: Peer Review and U. S. Science Policy , 1990 .

[6]  M. Mahoney Publication prejudices: An experimental study of confirmatory bias in the peer review system , 1977, Cognitive Therapy and Research.

[7]  Rosa Rodriguez-Sánchez,et al.  Adverse selection of reviewers , 2015, J. Assoc. Inf. Sci. Technol..

[8]  Cassidy R. Sugimoto,et al.  Bias in peer review , 2013, J. Assoc. Inf. Sci. Technol..

[9]  J. Burnham The evolution of editorial peer review. , 1990, JAMA.

[10]  Harald Merckelbach,et al.  Peer-Review: Let's Imitate the Lawyers! , 2002, Cortex.

[11]  Cassandra C. Elrod,et al.  Information Science and Technology , 2019, Washington Information Directory 2019–2020.

[12]  A. Kohn Why incentive plans cannot work , 1993 .

[13]  Patrick L. Warren,et al.  Optimal Agency Bias and Regulatory Review , 2013, The Journal of Legal Studies.

[14]  Rosa Rodriguez-Sánchez,et al.  The principal‐agent problem in peer review , 2015, J. Assoc. Inf. Sci. Technol..

[15]  Juan Miguel Campanario,et al.  Challenging Dominant Physics Paradigms , 2004 .

[16]  A. Kohn Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A'S, Praise, and Other Bribes , 1999 .

[17]  D. W. Fiske,et al.  But the Reviewers Are Making Different Criticisms of My Paper! Diversity and Uniqueness in Reviewer Comments. , 1990 .

[18]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 1 , 1998 .

[19]  E. Fehr,et al.  Psychological Foundations of Incentives , 2002 .

[20]  I. Sternlieb,et al.  Prevention of Wilson's disease in asymptomatic patients. , 1968, The New England journal of medicine.

[21]  Norman Kaplan,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1974 .