Author-Reviewer Homophily in Peer Review

The fairness of scholarly peer review has been challenged by evidence of disparities in publication outcomes based on author demographic characteristics. To assess this, we conducted an exploratory analysis of peer review outcomes of 23,876 initial submissions and 7,192 full submissions that were submitted to the biosciences journal eLife between 2012 and 2017. Women and authors from nations outside of North America and Europe were underrepresented both as gatekeepers (editors and peer reviewers) and authors. We found evidence of a homophilic relationship between the demographics of the gatekeepers and authors and the outcome of peer review; that is, there were higher rates of acceptance in the case of gender and country homophily. The acceptance rate for manuscripts with male last authors was seven percent, or 3.5 percentage points, greater than for female last authors (95% CI = [0 . 5 , 6 . 4]); this gender inequity was greatest, at nine percent or about 4.8 percentage points (95% CI = [0 . 3 , 9 . 1]), when the team of reviewers was all male; this difference was smaller and not significantly different for mixed-gender reviewer teams. Homogeny between countries of the gatekeeper and the corresponding author was also associated with higher acceptance rates for many countries. To test for the persistence controlling for many potentially confounding factors. These results provide evidence affirming observations from the univariate analysis in Fig 6.

[1]  F. Squazzoni,et al.  The effect of publishing peer review reports on referee behavior in five scholarly journals , 2019, Nature Communications.

[2]  Ran Blekhman,et al.  Tracking the popularity and outcomes of all bioRxiv preprints , 2019, bioRxiv.

[3]  Misha Teplitskiy,et al.  The sociology of scientific validity: How professional networks shape judgement in peer review , 2018, Research Policy.

[4]  Julia Schroeder,et al.  Gender differences in authorships are not associated with publication bias in an evolutionary journal , 2018, PloS one.

[5]  Christophe Bernard,et al.  Editorial: Gender Bias in Publishing: Double-Blind Reviewing as a Solution? , 2018, eNeuro.

[6]  A. van de Rijt,et al.  The Matthew effect in science funding , 2018, Proceedings of the National Academy of Sciences.

[7]  Robyn Tamblyn,et al.  Assessment of potential bias in research grant peer review in Canada , 2018, Canadian Medical Association Journal.

[8]  Y. Shoda,et al.  Persistent Underrepresentation of Women’s Science in High Profile Journals , 2018, bioRxiv.

[9]  Barbara McGillivray,et al.  Uptake and outcome of manuscripts in Nature journals by review model and author characteristics , 2018, Research Integrity and Peer Review.

[10]  D. Groneberg,et al.  Gender disparities in high-quality research revealed by Nature Index journals , 2018, PloS one.

[11]  H. Witteman,et al.  Female grant applicants are equally successful when peer reviewers assess the science, but not when they assess the scientist , 2017, bioRxiv.

[12]  Erin Hengel,et al.  Publishing while Female. Are women held to higher standards? Evidence from peer review. , 2017 .

[13]  Min Zhang,et al.  Reviewer bias in single- versus double-blind peer review , 2017, Proceedings of the National Academy of Sciences.

[14]  A. Darzi,et al.  Explicit Bias Toward High-Income-Country Research: A Randomized, Blinded, Crossover Experiment Of English Clinicians. , 2017, Health affairs.

[15]  S. R. King Consultative review is worth the wait , 2017, eLife.

[16]  Fei Shu,et al.  Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016) , 2017, Aslib J. Inf. Manag..

[17]  F. Squazzoni,et al.  Publishing: Journals could share peer-review data , 2017, Nature.

[18]  Sarah Huggett,et al.  Gender in the Global Research Landscape raw data , 2017 .

[19]  Johanna Espin,et al.  A persistent lack of international representation on editorial boards in environmental biology , 2017, bioRxiv.

[20]  V. Halloin,et al.  Gender and research funding success: Case of the Belgian F.R.S.-FNRS , 2017 .

[21]  Kim A. Weeden,et al.  Degrees of Difference: Gender Segregation of US Doctorates by Field and Program Prestige , 2017 .

[22]  Brooks Hanson,et al.  Journals invite too few women to referee , 2017, Nature.

[23]  S. Merchant,et al.  The Plant Cell Begins Opt-in Publishing of Peer Review Reports[OPEN] , 2016, Plant Cell.

[24]  M. Kocher,et al.  Single-blind vs Double-blind Peer Review in the Setting of Author Prestige. , 2016, JAMA.

[25]  Timothy Kassis,et al.  How do research faculty in the biosciences evaluate paper authorship criteria? , 2016, bioRxiv.

[26]  Vincent Larivière,et al.  Is Science Built on the Shoulders of Women? A Study of Gender Differences in Contributorship , 2016, Academic medicine : journal of the Association of American Medical Colleges.

[27]  Wei Yang,et al.  Policy: Boost basic research in China , 2016, Nature.

[28]  Shilad Sen,et al.  Gender Representation on Journal Editorial Boards in the Mathematical Sciences , 2016, PloS one.

[29]  Cassidy R. Sugimoto,et al.  Contributorship and division of labor in knowledge production , 2016, Social studies of science.

[30]  G. Pinholster,et al.  Journals and funders confront implicit bias in peer review , 2016 .

[31]  Andrew M. Collings,et al.  The effects of an editor serving as one of the reviewers during the peer-review process , 2016, F1000Research.

[32]  O. Pourquié,et al.  Future developments: your thoughts and our plans , 2016, Development.

[33]  N. Ellemers,et al.  Gender contributes to personal research funding success in The Netherlands , 2015, Proceedings of the National Academy of Sciences.

[34]  N. Zinovyeva,et al.  Does the Gender Composition of Scientific Committees Matter? , 2015, SSRN Electronic Journal.

[35]  Theodore Eugene Day,et al.  The big consequences of small biases: A simulation of peer review , 2015 .

[36]  Ricardo Conejo,et al.  Personal attributes of authors and reviewers, social bias and the outcomes of peer review: a case study , 2015, F1000Research.

[37]  A. Mulligan,et al.  Nature journals offer double-blind review , 2015, Nature.

[38]  Daniel B. Larremore,et al.  Systematic inequality and hierarchy in faculty hiring networks , 2015, Science Advances.

[39]  Claire Mathieu,et al.  Homophily and the Glass Ceiling Effect in Social Networks , 2015, ITCS.

[40]  Vincent Larivière,et al.  Team size matters: Collaboration and scientific impact since 1900 , 2014, J. Assoc. Inf. Sci. Technol..

[41]  Alexander M. Petersen,et al.  Inequality and cumulative advantage in science careers: a case study of high-impact journals , 2014, EPJ Data Science.

[42]  Cassidy R. Sugimoto,et al.  The reviewer in the mirror: examining gendered and ethnicized notions of reciprocity in peer review , 2014, Scientometrics.

[43]  Sarah J. Graves,et al.  Women are underrepresented on the editorial boards of journals in environmental biology and natural resource management , 2014, PeerJ.

[44]  Molly Carnes,et al.  Threats to objectivity in peer review: the case of gender. , 2014, Trends in pharmacological sciences.

[45]  Jason M. Sheltzer,et al.  Elite male faculty in the life sciences employ fewer women , 2014, Proceedings of the National Academy of Sciences.

[46]  Cassidy R. Sugimoto,et al.  Bibliometrics: Global gender disparities in science , 2013, Nature.

[47]  Lesley G. Campbell,et al.  Gender-Heterogeneous Working Groups Produce Higher Quality Science , 2013, PloS one.

[48]  R. Schekman,et al.  The eLife approach to peer review , 2013, eLife.

[49]  M. Bordons,et al.  Assessing gender balance among journal authors and editorial board members , 2013, Scientometrics.

[50]  Helen Shen Inequality quantified: Mind the gender gap , 2013, Nature.

[51]  Carl T. Bergstrom,et al.  The Role of Gender in Scholarly Authorship , 2012, PloS one.

[52]  Xiangyi Zhang,et al.  Effect of reviewer's origin on peer review: China vs. non‐China , 2012, Learn. Publ..

[53]  G. Bedi,et al.  Gender inequality in awarded research grants , 2012, The Lancet.

[54]  Karen Hapgood,et al.  The academic jungle: ecosystem modelling reveals why women are driven out of research , 2012 .

[55]  Michael Szell,et al.  How women organize social networks different from men , 2012, Scientific Reports.

[56]  Isabel Metz,et al.  An update of gender diversity in editorial boards: a longitudinal study of management journals , 2012 .

[57]  T. Pieber,et al.  Women underrepresented on editorial boards of 60 major medical journals. , 2011, Gender medicine.

[58]  Mary Stegmaier,et al.  Getting on the Board: The Presence of Women in Political Science Journal Editorial Positions , 2011, PS: Political Science & Politics.

[59]  Muriel Niederle,et al.  Gender and Competition , 2011 .

[60]  Laura Valkonen,et al.  Gender balance in Cortex acceptance rates , 2011, Cortex.

[61]  Hong Jiang,et al.  Sex Differences in Application, Success, and Funding Rates for NIH Extramural Programs , 2011, Academic medicine : journal of the Association of American Medical Colleges.

[62]  Rodrigo Costas,et al.  Do age and professional rank influence the order of authorship in scientific publications? Some evidence from a micro-level perspective , 2011, Scientometrics.

[63]  Liv Langfeldt,et al.  How Professors Think: Inside the Curious World of Academic Judgment , 2011 .

[64]  S. Ceci,et al.  Understanding current causes of women's underrepresentation in science , 2011, Proceedings of the National Academy of Sciences.

[65]  Bernd Pulverer,et al.  A transparent black box , 2010, The EMBO journal.

[66]  Rebecca S. Benner,et al.  Differences in editorial board reviewer behavior based on gender. , 2010, Journal of women's health.

[67]  Isabel Metz,et al.  Gender Diversity in Editorial Boards of Management Journals , 2009 .

[68]  Amber E. Budden,et al.  To Name or Not to Name: The Effect of Changing Author Gender on Peer Review , 2009 .

[69]  Elizabeth R. Ellwood,et al.  Do gender, nationality, or academic age affect review decisions? An analysis of submissions to the journal Biological Conservation , 2009 .

[70]  R. Primack,et al.  Bias in the review process , 2008 .

[71]  W. Heath The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies , 2008 .

[72]  Mark Ware,et al.  Peer review in scholarly journals: Perspective of the scholarly community - Results from an international study , 2008, Inf. Serv. Use.

[73]  Françoise Salager-Meyer,et al.  Scientific Publishing in Developing Countries: Challenges for the Future. , 2008 .

[74]  Anna Duszak,et al.  Publishing Academic Texts in English: A Polish Perspective. , 2008 .

[75]  K. Zou,et al.  Gender differences in research grant applications and funding outcomes for medical school faculty. , 2008, Journal of women's health.

[76]  Zosia Kmietowicz,et al.  Double blind peer reviews are fairer and more objective, say academics , 2008, BMJ : British Medical Journal.

[77]  S. Sonnad,et al.  Women on professional society and journal editorial boards. , 2007, Journal of the National Medical Association.

[78]  George Tomlinson,et al.  The Meaning of Author Order in Medical Research , 2007, Journal of Investigative Medicine.

[79]  M. Hochberg,et al.  Author Sequence and Credit for Contributions in Multiauthored Publications , 2007, PLoS biology.

[80]  Sara Schroter,et al.  Why do peer reviewers decline to review? A survey , 2006, Journal of Epidemiology and Community Health.

[81]  A. Gelman,et al.  The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant , 2006 .

[82]  C. Gross,et al.  Effect of blinded peer review on abstract acceptance. , 2006, JAMA.

[83]  James M Provenzale,et al.  Peer review at the American Journal of Roentgenology: how reviewer and manuscript characteristics affected editorial decisions on 196 major papers. , 2004, AJR. American journal of roentgenology.

[84]  D. King The scientific impact of nations , 2004, Nature.

[85]  Paola Villa,et al.  The Editorial Boards of Italian Economics Journals: Women, Gender, and Social Networking , 2003 .

[86]  Tom Tregenza,et al.  Gender bias in the refereeing process , 2002 .

[87]  E Pasini,et al.  Language and publication in "Cardiovascular Research" articles. , 2002, Cardiovascular research.

[88]  Jerry A. Jacobs,et al.  Gender and the Stratification of Colleges , 1999 .

[89]  Leandre R. Fabrigar,et al.  The Review Process at PSPB: Correlates of Interreviewer Agreement and Manuscript Acceptance , 1999 .

[90]  Yu Xie,et al.  Sex differences in research productivity : New evidence about an old puzzle , 1998 .

[91]  A. Link US and non-US submissions: an analysis of reviewer bias. , 1998, JAMA.

[92]  J. Grant,et al.  No evidence of sexism in peer review , 1997, Nature.

[93]  C. Wennerås,et al.  Nepotism and sexism in peer-review , 1997, Nature.

[94]  J. R. Gilbert,et al.  Is there gender bias in JAMA's peer review process? , 1994, JAMA.

[95]  Harold Maurice Collins,et al.  New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System , 1991 .

[96]  Margaret E. Lloyd,et al.  Gender factors in reviewer recommendations for manuscript publication. , 1990, Journal of applied behavior analysis.

[97]  Burt V. Bronk,et al.  Hierarchy of sciences , 1977 .

[98]  Jeffrey Pfeffer,et al.  Paradigm Development and Particularism: Journal Publication in Three Scientific Disciplines , 1977 .

[99]  M A Schnitker,et al.  THE AMERICAN FEDERATION FOR CLINICAL RESEARCH. , 1942, Science.

[100]  Nature’s under-representation of women , 2018, Nature.

[101]  Yoshinori Hatori,et al.  The Difference between Significant and Non-significant , 2017 .

[102]  Michaela Willi-Hooper,et al.  Gender bias in scholarly peer review , 2017 .

[103]  C. Sean Burns,et al.  Editor and reviewer gender influence the peer review process but not peer review outcomes at an ecology journal , 2016 .

[104]  Joon-Oh Park,et al.  The Increasing Dominance of Teams in Production of Knowledge , 2014 .

[105]  S. Adamo Attrition of Women in the Biological Sciences: Workload, Motherhood, and Other Explanations Revisited , 2013 .

[106]  Cassidy R. Sugimoto,et al.  Bias in peer review , 2013, J. Assoc. Inf. Sci. Technol..

[107]  H. Breithaupt Searching for discrimination Are women treated fairly in the EMBO postdoctoral fellowship scheme ? , 2013 .

[108]  Nigel W. Bond,et al.  Gender differences in peer reviews of grant applications: A substantive-methodological synergy in support of the null hypothesis model , 2011, J. Informetrics.

[109]  S. Ceci,et al.  Sex Differences in Math-Intensive Fields , 2010 .

[110]  Susan M. Barnett,et al.  Women's underrepresentation in science: sociocultural and biological considerations. , 2009, Psychological bulletin.

[111]  Lutz Bornmann,et al.  Gatekeepers of science - Effects of external reviewers' attributes on the assessments of fellowship applications , 2007, J. Informetrics.

[112]  Shelley J. Correll,et al.  Expectation states theory. , 2006 .

[113]  Women in neuroscience: a numbers game , 2006, Nature Neuroscience.

[114]  Joel Podolny Status Signals: A Sociological Study of Market Competition , 2005 .

[115]  R. Kostoff The (scientific) wealth of nations , 2004 .

[116]  J. S. Long,et al.  Scientific Careers: Universalism and Particularism , 1995 .

[117]  R. Blank The Effects of Double-Blind versus Single-Blind Reviewing: Experimental Evidence from The American Economic Review , 1991 .

[118]  William E. Winkler,et al.  String Comparator Metrics and Enhanced Decision Rules in the Fellegi-Sunter Model of Record Linkage. , 1990 .

[119]  Joseph Berger,et al.  Status Characteristics and Social Interaction , 1972 .

[120]  R. Merton The Matthew effect in science. The reward and communication systems of science are considered. , 1968, Science.