Peer review motivation frames: A qualitative approach

Abstract There is an ongoing need to support high-quality research publications that requires a greater emphasis on the role of the peer review process. The difficulties faced by editors in finding committed reviewers and in avoiding delayed review reports, as well as the frequency of failure in manuscript error detection, all stress the need to identify incentive strategies that will ensure high-quality peer reviews. Based on a qualitative approach, this paper explores referees' decision frames when reviewing, the characteristics of the review behaviour, and the associated benefits and costs. Semi-structured interviews were conducted with 42 journal referees. The results highlight the motivating factors that affect the decision to review, or not to review. Two motivation frames-of-reference were identified: that of a prospective member of the scientific community focused on self-achievement vs. that of a member of the scientific community focused on the group. Different situational cues activate a particular frame: the match between reviewer's expertise and the manuscript topic, the identification with the scientific community, and the quality of the journal. The findings suggest strategies able to minimize referees' perceived costs when reviewing. This research sheds new light on the strategies that have the potential to boost the peer review process.

[1]  Ching-chong Lai,et al.  Is It Worthwhile to Pay Referees , 2001 .

[2]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[3]  Nikolaus Kriegeskorte,et al.  Beyond open access: visions for open evaluation of scientific papers by post-publication peer review , 2012 .

[4]  Roderick M. Kramer,et al.  Choice behavior in social dilemmas: Effects of social identity, group size, and decision framing. , 1986 .

[5]  Kjell Goldmann.  Appropriateness and Consequences: The Logic of Neo‐Institutionalism , 2005 .

[6]  B. Frey,et al.  Motivation crowding theory , 2001 .

[7]  Laurel L. Haak,et al.  Race, Ethnicity, and NIH Research Awards , 2011, Science.

[8]  Joshua S. Gans,et al.  Why Referees Are Not Paid (Enough) , 1998 .

[9]  J. Armstrong,et al.  Peer review for journals: Evidence on quality control, fairness, and innovation , 1997 .

[10]  C. Wennerås,et al.  Nepotism and sexism in peer-review , 1997, Nature.

[11]  David J. Ketchen,et al.  From The Editors Volunteer and Shirking Behaviors Among The Daca , 2008 .

[12]  David De Cremer,et al.  Social value orientations and the strategic use of fairness in ultimatum bargaining , 2004 .

[13]  Sara Schroter,et al.  Why do peer reviewers decline to review? A survey , 2006, Journal of Epidemiology and Community Health.

[14]  Ulf Sandström,et al.  Persistent nepotism in peer-review , 2008, Scientometrics.

[15]  M. Lepper,et al.  The Hidden costs of reward : new perspectives on the psychology of human motivation , 1978 .

[16]  J. K. Murnighan,et al.  The volunteer dilemma , 1993 .

[17]  Max H. Bazerman,et al.  Egocentric Interpretations of Fairness in Asymmetric, Environmental Social Dilemmas: Explaining Harvesting Behavior and the Role of Communication , 1996 .

[18]  M. Patton,et al.  Qualitative evaluation and research methods , 1992 .

[19]  Razvan V. Florian.  Aggregating post-publication peer reviews and ratings , 2012, Front. Comput. Neurosci..

[20]  Gary D. Thompson,et al.  Does Paying Referees Expedite Reviews?: Results of a Natural Experiment , 2010 .

[21]  Ofer H. Azar.  The Academic Review Process: How Can We Make it More Efficient? , 2005 .

[22]  P. Rothwell,et al.  Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? , 2000, Brain : a journal of neurology.

[23]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[24]  Bradley M. Hemminger,et al.  Decoupling the scholarly journal , 2011, Front. Comput. Neurosci..

[25]  Linda Klebe Trevino,et al.  Editor's Comments: Why Review? Because Reviewing is a Professional Responsibility , 2008 .

[26]  Ann E. Tenbrunsel,et al.  Effective Matrices, Decision Frames, and Cooperation in Volunteer Dilemmas: A Theoretical Perspective on Academic Peer Review , 2011, Organ. Sci..

[27]  Sharan B. Merriam,et al.  Qualitative Research: A Guide to Design and Implementation , 2009 .

[28]  H. Bernard,et al.  Data Management and Analysis Methods , 2000 .

[29]  G. Northcraft,et al.  In the eye of the beholder: Payoff structures and decision frames in social dilemmas , 2009 .

[30]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[31]  Raj Chetty,et al.  What Policies Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics , 2014 .

[32]  Daniel S. Hamermesh,et al.  Facts and Myths about Refereeing , 1994 .

[33]  Madan M. Pillutla,et al.  Unfairness, Anger, and Spite: Emotional Rejections of Ultimatum Offers , 1996 .

[34]  Young Hoan Cho,et al.  Peer reviewers learn from giving comments , 2011 .

[35]  Richard E. Boyatzis,et al.  Transforming Qualitative Information: Thematic Analysis and Code Development , 1998 .

[36]  D. Messick,et al.  Sanctioning Systems, Decision Frames, and Cooperation , 1999 .

[37]  James G. March,et al.  A primer on decision making : how decisions happen , 1994 .

[38]  Mason A. Carpenter,et al.  Editor's Comments: Mentoring Colleagues in the Craft and Spirit of Peer Review , 2009 .

[39]  Stacy M Carter,et al.  Journal peer review in context: A qualitative study of the social and subjective dimensions of manuscript review in biomedical publishing. , 2011, Social science & medicine.

[40]  Juan Miguel Campanario,et al.  Have Referees Rejected Some of the Most-Cited Articles of All Times? , 1996, J. Am. Soc. Inf. Sci..

[41]  Michel J. J. Handgraaf,et al.  Less power or powerless? Egocentric empathy gaps and the irony of having little versus no power in social decision making. , 2008, Journal of personality and social psychology.

[42]  M. Engle.  Book Review: Qualitative Data Analysis: An Expanded Sourcebook (2nd Ed.) , 1999 .

[43]  Sara Schroter,et al.  Surveys of current status in biomedical science grant review: funding organisations' and grant reviewers' perspectives , 2010, BMC medicine.

[44]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[45]  A. Rustichini,et al.  Pay Enough or Don't Pay at All , 2000 .

[46]  Richard Horton,et al.  Is peer review in crisis ? , 2004 .

[47]  F. Godlee,et al.  Effect on the quality of peer review of blinding reviewers and asking them to sign their reports: a randomized controlled trial. , 1998, JAMA.

[48]  D. Shatz.  Peer Review: A Critical Inquiry , 2004 .

[49]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[50]  Molly Carnes,et al.  Threats to objectivity in peer review: the case of gender. , 2014, Trends in pharmacological sciences.

[51]  J F Waeckerle,et al.  Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. , 1998, Annals of emergency medicine.

[52]  T. Jefferson,et al.  Editorial peer review for improving the quality of reports of biomedical studies. , 2007, The Cochrane database of systematic reviews.

[53]  M. Brewer,et al.  Effects of group identity on resource use in a simulated commons dilemma. , 1984, Journal of personality and social psychology.