Skin in the Game: Personal Accountability and Journal Peer Review

Two preregistered studies explored the likelihood paper reviewers would request clarification from authors regarding potential questionable research practices (QRPs). Study 1 participants were instructed to imagine reviewing a journal manuscript as either a coauthor or peer reviewer and rate the extent to which they would request clarification from the author when encountering potential QRPs. Participants reported greater likelihood of requesting clarification when assigned to the coauthor relative to the peer reviewer role. Study 2 participants were assigned to either an anonymous or open-review condition and rated the extent to which they would seek clarification from an author regarding potential QRPs. Men (but not women) in the open review condition reported greater likelihood of seeking clarification about potential QRPs than men in the blind review condition. Results provide tentative evidence that motivational factors influence the peer review process, and suggestions are made for improving peer review practices.

[1]  E. Higgins,et al.  Ideal versus ought predilections for approach and avoidance: distinct self-regulatory systems. , 1994, Journal of personality and social psychology.

[2]  Jaime A. Teixeira da Silva,et al.  Challenges to open peer review , 2019, Online Inf. Rev..

[3]  Nikolaus Kriegeskorte,et al.  Open Evaluation: A Vision for Entirely Transparent Post-Publication Peer Review and Rating for Science , 2012, Front. Comput. Neurosci..

[4]  Mitch Brown,et al.  In Defense of the Questionable: Defining the Basis of Research Scientists’ Engagement in Questionable Research Practices , 2018, Journal of empirical research on human research ethics : JERHRE.

[5]  M. Hojat,et al.  Impartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review Process , 2003, Advances in health sciences education : theory and practice.

[6]  Andreas Neef,et al.  Gender bias in scholarly peer review , 2017, eLife.

[7]  Chao-Min Chiu,et al.  Exploring and mitigating social loafing in online communities , 2010, Comput. Hum. Behav..

[8]  Nina Mazar,et al.  The Dishonesty of Honest People: A Theory of Self-Concept Maintenance , 2008 .

[9]  F. Godlee.  Making reviewers visible: openness, accountability, and credit. , 2002, JAMA.

[10]  Mitch Brown,et al.  Ambiguity : Justifiable Bases for Engaging in Questionable Research Practices , 2018 .

[11]  M. Mahoney.  Publication prejudices: An experimental study of confirmatory bias in the peer review system , 1977, Cognitive Therapy and Research.

[12]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[13]  Francisco Segado-Boj,et al.  Attitudes toward Open Access, Open Peer Review, and Altmetrics among Contributors to Spanish Scholarly Journals , 2018, Journal of Scholarly Publishing.

[14]  Samuel V. Bruton,et al.  Ethical Consistency and Experience: An Attempt to Influence Researcher Attitudes Toward Questionable Research Practices Through Reading Prompts , 2019, Journal of empirical research on human research ethics : JERHRE.

[15]  K. White,et al.  Good and Guilt-Free: The Role of Self-Accountability in Influencing Preferences for Products with Ethical Attributes , 2013 .

[16]  Sara Schroter,et al.  Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. , 2006, JAMA.

[17]  D. Sacco,et al.  Assessing the Efficacy of a Training Intervention to Reduce Acceptance of Questionable Research Practices in Psychology Graduate Students , 2019, Journal of empirical research on human research ethics : JERHRE.

[18]  T. Tregenza,et al.  Double-blind review favours increased representation of female authors. , 2008, Trends in ecology & evolution.

[19]  Vladas Griskevicius,et al.  Blatant benevolence and conspicuous consumption: when romantic motives elicit strategic costly signals. , 2007, Journal of personality and social psychology.

[20]  Vladas Griskevicius,et al.  Peacocks, Porsches, and Thorstein Veblen: conspicuous consumption as a sexual signaling system. , 2011, Journal of personality and social psychology.

[21]  K. Williams,et al.  Identifiability as a deterrant to social loafing: Two cheering experiments. , 1981 .

[22]  Birgit Schmidt,et al.  Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers , 2017, PloS one.

[23]  Christopher J. Bryan,et al.  When cheating would make you a cheater: implicating the self prevents unethical behavior. , 2013, Journal of experimental psychology. General.

[24]  Rosa Rodriguez-Sánchez,et al.  The principal‐agent problem in peer review , 2015, J. Assoc. Inf. Sci. Technol..

[25]  N. Kerr.  HARKing: Hypothesizing After the Results are Known , 1998, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[26]  Christina M. Brown,et al.  Discrepancy-based and anticipated emotions in behavioral self-regulation. , 2011, Emotion.

[27]  J. Bohannon.  Who's afraid of peer review? , 2013, Science.

[28]  C. Haug,et al.  Peer-Review Fraud--Hacking the Scientific Publication Process. , 2015, The New England journal of medicine.

[29]  M. A. Hadi.  Fake peer‐review in research publication: revisiting research purpose and academic integrity , 2016, The International journal of pharmacy practice.

[30]  D. Paulhus.  Two-component models of socially desirable responding. , 1984 .

[31]  Richard Smith,et al.  Peer Review: A Flawed Process at the Heart of Science and Journals , 2006 .

[32]  Pat Barclay.  Altruism as a courtship display: some effects of third-party generosity on audience perceptions. , 2010, British journal of psychology.

[33]  F. Dudbridge,et al.  Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models , 2015, BMJ Open.

[34]  R. Whittaker,et al.  Journal review and gender equality: a critical comment on Budden et al. , 2008, Trends in ecology & evolution.

[35]  K. Williams,et al.  Many Hands Make Light the Work: The Causes and Consequences of Social Loafing , 1979 .

[36]  G. Loewenstein,et al.  Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling , 2012, Psychological science.

[37]  Stuart A. Kirk,et al.  Recognition of Authors in Blind Review of Manuscripts , 1981 .

[38]  Gerald Häubl,et al.  The Signature Effect: Signing Influences Consumption-Related Behavior by Priming Self-Identity , 2011 .

[39]  Wendy Iredale,et al.  Men behaving nicely: public goods as peacock tails. , 2013, British journal of psychology.

[40]  Nina Mazar,et al.  Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end , 2012, Proceedings of the National Academy of Sciences.