The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future

Review, promotion, and tenure (RPT) processes significantly affect how faculty direct their own career and scholarly progression. Although RPT practices vary between and within institutions, and affect various disciplines, ranks, institution types, genders, and ethnicity in different ways, some consistent themes emerge when investigating what faculty would like to change about RPT. For instance, over the last few decades, RPT processes have generally increased the value placed on research, at the expense of teaching and service, which often results in an incongruity between how faculty actually spend their time vs. what is considered in their evaluation. Another issue relates to publication practices: most agree RPT requirements should encourage peer-reviewed works of high quality, but in practice, the value of publications is often assessed using shortcuts such as the prestige of the publication venue, rather than on the quality and rigor of peer review of each individual item. Open access and online publishing have made these issues even murkier due to misconceptions about peer review practices and concerns about predatory online publishers, which leaves traditional publishing formats the most desired despite their restricted circulation. And, efforts to replace journal-level measures such as the impact factor with more precise article-level metrics (e.g., citation counts and altmetrics) have been slow to integrate with the RPT process. Questions remain as to whether, or how, RPT practices should be changed to better reflect faculty work patterns and reduce pressure to publish in only the most prestigious traditional formats. To determine the most useful way to change RPT, we need to assess further the needs and perceptions of faculty and administrators, and gain a better understanding of the level of influence of written RPT guidelines and policy in an often vague process that is meant to allow for flexibility in assessing individuals.

[1]  David J. Prottas,et al.  Relationships Among Faculty Perceptions of Their Tenure Process and Their Commitment and Engagement , 2017 .

[2]  James S. Fairweather,et al.  Faculty reward structures: Toward institutional and professional homogenization , 1993 .

[3]  M. Seipel,et al.  Assessing Publication for Tenure , 2003 .

[4]  Peter Suber Thoughts on prestige, quality, and open access , 2010 .

[5]  Stephen Curry Let's move beyond the rhetoric: it's time to change how we judge research. , 2018 .

[6]  Sudeep Sarkar,et al.  Changing the academic culture: Valuing patents and commercialization toward tenure and career advancement , 2014, Proceedings of the National Academy of Sciences.

[7]  Linda K. Johnsrud,et al.  Barriers to Tenure for Women and Minorities , 2017 .

[8]  E. Garfield Journal impact factor: a brief review. , 1999, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[9]  R. Green,et al.  THE SECOND DECADE OF THE FACULTY PUBLICATION PROJECT: JOURNAL ARTICLE PUBLICATIONS AND THE IMPORTANCE OF FACULTY SCHOLARSHIP , 2007 .

[10]  Vincent Larivière,et al.  The Use of Bibliometrics for Assessing Research: Possibilities, Limitations and Adverse Effects , 2015 .

[11]  Executive Summary,et al.  Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and Needs in Seven Disciplines - Executive Summary , 2010 .

[12]  B. Brembs Prestigious Science Journals Struggle to Reach Even Average Reliability , 2018, Front. Hum. Neurosci..

[13]  Björn Brembs,et al.  Deep impact: unintended consequences of journal rank , 2013, Front. Hum. Neurosci..

[14]  Hewitt W. Matthews,et al.  Barriers to scholarship in dentistry, medicine, nursing, and pharmacy practice faculty. , 2007, American journal of pharmaceutical education.

[15]  W. Edwards,et al.  African American faculty expressing concerns: breaking the silence at predominantly white research oriented universities , 2016 .

[16]  David H. Sinason,et al.  Pitfalls of Using Citation Indices for Making Academic Accounting Promotion, Tenure, Teaching Load, and Merit Pay Decisions , 2011 .

[17]  Karen Whalen,et al.  Perceptions of Tenure and Tenure Reform in Academic Pharmacy , 2014, American Journal of Pharmaceutical Education.

[18]  M. Mahoney,et al.  Open exchange and epistemic progress. , 1985 .

[19]  Christina K. Pikas,et al.  A multi-disciplinary perspective on emergent and future innovations in peer review , 2017, F1000Research.

[20]  Blaise Cronin,et al.  E-Journals and Tenure , 1995, J. Am. Soc. Inf. Sci..

[21]  Peter S. Li,et al.  The Equity Myth: Racialization and Indigeneity at Canadian Universities , 2017 .

[22]  Janet Dagenais Brown Citation searching for tenure and promotion: an overview of issues and tools , 2014 .

[23]  Stephen Curry,et al.  Let’s move beyond the rhetoric: it’s time to change how we judge research , 2018, Nature.

[24]  Tony Ross-Hellauer,et al.  What is open peer review? A systematic review , 2017, F1000Research.

[25]  Jeremy W. Fox Can blogging change how ecologists share ideas? In economics, it already has. , 2012 .

[26]  Robert M. Diamond,et al.  Changing Priorities at Research Universities, 1991-1996. Based on: The National Study of Research Universities on the Balance between Research and Undergraduate Teaching (1992), by Peter J. Gray, Robert C. Froh, Robert M. Diammond. , 1998 .

[27]  B. Malsch,et al.  Journal ranking effects on junior academics: Identity fragmentation and politicization , 2015 .

[28]  Benedikt V. Ehinger,et al.  Faculty Opinions recommendation of PSYCHOLOGY. Estimating the reproducibility of psychological science. , 2015 .

[29]  Brett Thomas Buttliere,et al.  Using science and psychology to improve the dissemination and evaluation of scientific work , 2014, Front. Comput. Neurosci..

[30]  Ted I. K. Youn,et al.  Learning from the Experience of Others: The Evolution of Faculty Tenure and Promotion Rules in Comprehensive Institutions , 2009 .

[31]  Brian A. Nosek,et al.  How open science helps researchers succeed , 2016, eLife.

[32]  Jennifer J. Otten,et al.  Getting Research to the Policy Table: A Qualitative Study With Public Health Researchers on Engaging With Policy Makers , 2015, Preventing chronic disease.

[33]  S. Haustein,et al.  Authorship, citations, acknowledgments and visibility in social media: Symbolic capital in the multifaceted reward system of science , 2018 .

[34]  C. Feghali-Bostwick,et al.  Gender Disparities in Faculty Rank: Factors that Affect Advancement of Women Scientists at Academic Medical Centers , 2018 .

[35]  Gaines H. Liner,et al.  Research requirements for promotion and tenure at PhD granting departments of economics , 2009 .

[36]  R. Schonfeld,et al.  Ithaka S+R US Faculty Survey 2015 , 2016 .

[37]  Susan K. Gardner,et al.  Evincing the Ratchet: A Thematic Analysis of the Promotion and Tenure Guidelines at a Striving University , 2014 .

[38]  S. Acker,et al.  Discipline and Publish: The Tenure Review Process in Ontario Universities , 2016 .

[39]  Mary Anne Holmes,et al.  What Does It Take to Get Tenure , 2004 .

[40]  Relative Importance of Performance Criteria in Promotion and Tenure Decisions: Perceptions of Pharmacy Faculty Members , 1995 .

[41]  R. Menges,et al.  Barriers to the Progress of Women and Minority Faculty , 1983 .

[42]  K. O’Meara,et al.  Advancing Engaged Scholarship in Promotion and Tenure: A Roadmap and Call for Reform. , 2015 .

[43]  R. Dellavalle,et al.  The write position , 2007, EMBO reports.

[44]  Victor M. H. Borden,et al.  Faculty Service Loads and Gender: Are Women Taking Care of the Academic Family? , 2017, SSRN Electronic Journal.

[45]  Vincent Larivière,et al.  The Journal Impact Factor: A brief history, critique, and discussion of adverse effects , 2018, Springer Handbook of Science and Technology Indicators.

[46]  T. L. Tang,et al.  Attitudes Toward Research and Teaching: Differences Between Administrators and Faculty Members. , 1997 .

[47]  Juan Pablo Alperin,et al.  How significant are the public dimensions of faculty work in review, promotion and tenure documents? , 2019, eLife.

[48]  B. Jordan,et al.  More Than Likes and Tweets: Creating Social Media Portfolios for Academic Promotion and Tenure. , 2017, Journal of graduate medical education.

[49]  Richard Smith,et al.  Peer Review: A Flawed Process at the Heart of Science and Journals , 2006 .

[50]  Samuel A. Moore,et al.  Erratum: “Excellence R Us”: university research and the fetishisation of excellence , 2017, Palgrave Communications.

[51]  Heather A. Piwowar,et al.  Altmetrics: Value all research products , 2013, Nature.

[52]  Anatoliy A. Gruzd,et al.  Tenure and promotion in the age of online social media , 2011, ASIST.

[53]  Bruce Macfarlane,et al.  Defining and Rewarding Academic Citizenship: The implications for university promotions policy , 2007 .

[54]  Ernest L. Boyer,et al.  The Scholarship of Engagement. , 1996 .

[55]  Emily S. Darling,et al.  The role of twitter in the life cycle of a scientific publication , 2013, ArXiv.

[56]  A. Welton,et al.  Assistant professors of color confront the inequitable terrain of academia: a community cultural wealth perspective , 2017 .

[57]  A. Goldstein,et al.  Community engagement in US and Canadian medical schools , 2011, Advances in medical education and practice.

[58]  B. Montgomery,et al.  Retention of Underrepresented Minority Faculty: Strategic Initiatives for Institutional Value Proposition Based on Perspectives from a Range of Academic Institutions. , 2015, Journal of undergraduate neuroscience education : JUNE : a publication of FUN, Faculty for Undergraduate Neuroscience.

[59]  D. C. May THE NATURE OF SCHOOL OF EDUCATION FACULTY WORK AND MATERIALS FOR PROMOTION AND TENURE AT A MAJOR RESEARCH UNIVERSITY , 2005 .

[60]  L. Claxton,et al.  Scientific authorship. Part 1. A window into scientific fraud? , 2005, Mutation research.

[61]  Hendrik P. van Dalen,et al.  Intended and Unintended Consequences of a Publish-or-Perish Culture: A Worldwide Survey , 2012, J. Assoc. Inf. Sci. Technol..

[62]  D. Lee,et al.  Beyond Teaching and Research: Faculty Perceptions of Service Roles at Research Universities , 2016 .

[63]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[64]  M. Wacha,et al.  The State of OA: A large-scale analysis of the prevalence and impact of Open Access articles , 2017 .

[65]  Diane Harley,et al.  Credit, time, and personality: The human challenges to sharing scholarly work using Web 2.0 , 2013, New Media Soc..

[66]  Peter Taylor,et al.  Citation Statistics , 2009, ArXiv.

[67]  Gary F. Peters,et al.  The Perceived Impact of Journal of Information Systems on Promotion and Tenure , 2015, J. Inf. Syst..

[68]  Julie Ellison,et al.  Scholarship in Public: Knowledge Creation and Tenure Policy in the Engaged University , 2008 .

[69]  M. Soares Collaborative research in light of the prevailing criteria for promotion and tenure in academia. , 2015, Genomics.

[70]  Nadia I. Awad,et al.  The Scholarly Merit of Social Media Use Among Clinical Faculty , 2014, The Journal of pharmacy technology : jPT : official publication of the Association of Pharmacy Technicians.

[71]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics , 2015, Nature.

[72]  Harriet Zuckerman,et al.  Age, aging, and age structure in science , 1968 .

[73]  Hude Quan,et al.  Authors' opinions on publication in relation to annual performance assessment , 2010, BMC medical education.

[74]  R. Cagan The San Francisco Declaration on Research Assessment , 2013, Disease Models & Mechanisms.

[75]  Robert G. Green,et al.  TENURE AND PROMOTION DECISIONS: THE RELATIVE IMPORTANCE OF TEACHING, SCHOLARSHIP, AND SERVICE , 2008 .

[76]  Anton J. Nederhof,et al.  Policy impact of bibliometric rankings of research performance of departments and individuals in economics , 2008, Scientometrics.

[77]  Anthony R. Hendrickson,et al.  Research Commentary. Academic Rewards for Teaching, Research, and Service: Data and Discourse , 1999, Inf. Syst. Res..

[78]  Sanford G. Thatcher The challenge of open access for university presses , 2007, Learn. Publ..

[79]  Jonathan P. Tennant,et al.  The state of the art in peer review , 2018, FEMS microbiology letters.

[80]  Mark A. Fuller,et al.  Research Standards for Promotion and Tenure in Information Systems , 2006, MIS Q..

[81]  C. J. King,et al.  SCHOLARLY COMMUNICATION: ACADEMIC VALUES AND SUSTAINABLE MODELS , 2006 .