Competing for impact and prestige: Deciphering the “alphabet soup” of academic publications and faculty productivity metrics

Accurate quantification of scholarly productivity continues to pose a significant challenge to academic medical institutions seeking to standardize faculty performance metrics. Numerous approaches have been described in this domain, from subjective measures employed in the past to rapidly evolving objective assessments of today. Metrics based on publication characteristics include a variety of easily categorized, normalized, referenced, and quantifiable data points. In general, such measures can be broadly grouped as being author-, manuscript-, and publication/journal-specific. Commonly employed units of measurement are derived from the number of publications and/or citations, in various combinations and derivations. In aggregate, these metrics are utilized to more objectively assess academic productivity, mainly for the purpose of determining faculty promotion and tenure potential; evaluating grant application/renewal competitiveness; journal/publication, and institutional benchmarking; faculty recruitment, retention, and placement; as well as various departmental and institutional performance assessments. This article provides an overview of different measures of academic productivity and scientific impact, focusing on bibliometric data utilization, including advantages and disadvantages of each respective methodological approach. The following core competencies are addressed in this article: Interpersonal skills and communication, practice-based learning and improvement, systems-based practice.

[1]  Carl T. Bergstrom,et al.  The Eigenfactor™ Metrics , 2008, The Journal of Neuroscience.

[2]  Garehatty Rudrappa Kanthraj,et al.  Impact Factor , 2019, IEEE Geoscience and Remote Sensing Magazine.

[3]  David F Kallmes,et al.  Is the h-index predictive of greater NIH funding success among academic radiologists? , 2011, Academic radiology.

[4]  Chun-Ting Zhang,et al.  Relationship of the h-index, g-index, and e-index , 2010, J. Assoc. Inf. Sci. Technol..

[5]  E. Garfield Use of Journal Citation Reports and Journal Performance Indicators in measuring short and long term journal impact. , 2000, Croatian medical journal.

[6]  G. Holden,et al.  Bibliometrics , 2005, Social work in health care.

[7]  Michel Zitt,et al.  Modifying the journal impact factor by fractional citation weighting: The audience factor , 2008, J. Assoc. Inf. Sci. Technol..

[8]  Lucille A. Carver,et al.  Recognizing diverse forms of scholarship in the modern medical college , 2013 .

[9]  W. Mallon,et al.  The continued evolution of faculty appointment and tenure policies at U.S. medical schools. , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[10]  P. Ramsden Describing and explaining research productivity , 1994 .

[11]  Nicholas E Kman,et al.  How we use social media to supplement a novel curriculum in medical education , 2012, Medical teacher.

[12]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[13]  Marek Kosmulski,et al.  MAXPROD - A New Index for assessment of the scientific output of an individual, and a comparison with the h-index , 2007 .

[14]  Wolfgang Glänzel,et al.  Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies , 2004, Scientometrics.

[15]  Johannes Weber,et al.  Publish or perish: a systematic review of interventions to increase academic publication rates , 2006 .

[16]  Henry L. Roediger,et al.  The h Index in Science: A New Measure of Scholarly Contribution , 2006 .

[17]  M. W. de Jong‐Hofman Comparison of selecting, abstracting and indexing by COMPENDEX, INSPEC and PASCAL and the impact of this on manual and automated retrieval of information , 1981 .

[18]  Stephanie J Bird,et al.  Self-plagiarism and dual and redundant publications: What is the problem? , 2002, Science and engineering ethics.

[19]  B. Medhi,et al.  Impact of the impact factor in biomedical research: its use and misuse. , 2009, Singapore medical journal.

[20]  Ignacio Roa,et al.  Comparison between Impact Factor, Eigenfactor Score, and SCImago Journal Rank Indicator in Anatomy and Morphology Journals Comparación entre Factor de Impacto, Eigenfactor Score e Indicador SCImago Journal Rank en Revistas de Anatomía y Morfología , 2015 .

[21]  E. Krupat,et al.  Changing the Culture of Academic Medicine: The C-Change Learning Action Network and Its Impact at Participating Medical Schools , 2013, Academic medicine : journal of the Association of American Medical Colleges.

[22]  D. Sin,et al.  Integrative Approach to Quality Assessment of Medical Journals Using Impact Factor, Eigenfactor, and Article Influence Scores , 2010, PloS one.

[23]  Lei Wang,et al.  Three options for citation tracking: Google Scholar, Scopus and Web of Science , 2006, Biomedical digital libraries.

[24]  Cathy C Sarli,et al.  Using publication metrics to highlight academic productivity and research impact. , 2014, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[25]  Ronald Rousseau,et al.  The immediacy index and the journal impact factor: two highly correlated derived measures , 2004 .

[26]  Heather A. Piwowar,et al.  The Imperative for Open Altmetrics , 2014 .

[27]  Michael G Crisp Eigenfactor , 2008 .

[28]  R. Rousseau,et al.  The R- and AR-indices: Complementing the h-index , 2007 .

[29]  Ashish Sham Nichani Whose manuscript is it anyway? The ‘Write’ position and number of authors… , 2013, Journal of Indian Society of Periodontology.

[30]  Chris Maloney,et al.  PubMed Central , 2017 .

[31]  Francisco Herrera,et al.  q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core , 2010, J. Informetrics.

[32]  Wolfgang Glänzel,et al.  On the Opportunities and Limitations of the H-index , 2006 .

[33]  Michael Schreiber,et al.  A modification of the h-index: The hm-index accounts for multi-authored manuscripts , 2008, J. Informetrics.

[34]  W. B. Fye Medical authorship: traditions, trends, and tribulations. , 1990, Annals of internal medicine.

[35]  Johan Bollen,et al.  A Principal Component Analysis of 39 Scientific Impact Measures , 2009, PloS one.

[36]  Charles R. Thomas,et al.  Bibliometric analysis of radiation oncology departmental scholarly publication productivity at domestic residency training institutions. , 2009, Journal of the American College of Radiology : JACR.

[37]  A. Mohamed,et al.  Publish or Perish: Evaluating and Promoting Scholarly Output. , 2013 .

[38]  R. Bast,et al.  How will we recruit, train, and retain physicians and scientists to conduct translational cancer research? , 2015, Cancer.

[39]  R. Green FACULTY RANK, EFFORT, AND SUCCESS: A STUDY OF PUBLICATION IN PROFESSIONAL JOURNALS , 1998 .

[40]  Xiaotian Chen,et al.  Cross-Examining Google Scholar , 2013 .

[41]  Martin Fenner,et al.  Altmetrics in Evolution: Defining and Redefining the Ontology of Article-Level Metrics , 2013 .

[42]  B. Alberts Impact Factor Distortions , 2013, Science.

[43]  R. E. Burton,et al.  The “half‐life” of some scientific and technical literatures , 1960 .

[44]  Aristides Gionis,et al.  Estimating Number of Citations Using Author Reputation , 2007, SPIRE.

[45]  Susan Bonzi,et al.  Motivations for citation: A comparison of self citation and citation to others , 1991, Scientometrics.

[46]  Eigenfactor TM Score and Article Influence TM Score : Detailed methods , .

[47]  Paula Younger,et al.  When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. , 2009, Health information and libraries journal.

[48]  Teodor P. Grantcharov,et al.  The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. , 2013, Surgery.

[49]  M. Jennions,et al.  The h index and career assessment by numbers. , 2006, Trends in ecology & evolution.

[50]  D. Cone,et al.  Measuring the measurable: a commentary on impact factor. , 2012, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[51]  Marian Thunnissen,et al.  Talent management in academia: performance systems and HRM policies , 2013 .

[52]  S. Redner How popular is your paper? An empirical study of the citation distribution , 1998, cond-mat/9804163.

[53]  Gangan Prathap,et al.  The 100 most prolific economists using the p-index , 2010, Scientometrics.

[54]  Leo Egghe,et al.  Dynamic h-index: The Hirsch index in function of time , 2007, J. Assoc. Inf. Sci. Technol..

[55]  L. Beckett,et al.  Improving Knowledge, Awareness, and Use of Flexible Career Policies Through an Accelerator Intervention at the University of California, Davis, School of Medicine , 2013, Academic medicine : journal of the Association of American Medical Colleges.

[56]  J. Roman,et al.  Recruitment and retention in academic medicine--what junior faculty and trainees want department chairs to know. , 2012, The American journal of the medical sciences.

[57]  L. Egghe An improvement of the h-index: the g-index , 2006 .

[58]  Qiang Ma,et al.  Socializing the h-index , 2013, J. Informetrics.

[59]  Nicolás Robinson-García,et al.  Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting , 2012, ArXiv.

[60]  J. Pandit Measuring academic productivity: don’t drop your ‘h’s! * , 2011, Anaesthesia.

[61]  Vincent Larivière,et al.  Benchmarking scientific output in the social sciences and humanities: The limits of existing databases , 2006, Scientometrics.

[62]  Marion Campbell,et al.  Use of the h-index to measure the quality of the output of health services researchers , 2014, Journal of health services research & policy.

[63]  Yang Liu,et al.  A bibliometric analysis of academic publication and NIH funding , 2013, J. Informetrics.

[64]  João Carlos Correia Baptista Soares de Mello,et al.  Comparative analysis of some individual bibliometric indices when applied to groups of researchers , 2014, Scientometrics.

[65]  Carl T. Bergstrom,et al.  Assessing citations with the Eigenfactor™ Metrics , 2008, Neurology.

[66]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[67]  Holly M. Bik,et al.  An Introduction to Social Media for Scientists , 2013, PLoS biology.

[68]  Carl T. Bergstrom,et al.  Differences in impact factor across fields and over time , 2008, J. Assoc. Inf. Sci. Technol..

[69]  Tan Yong-qin Individualized Service of the EBSCOhost Full Text Database , 2005 .

[70]  Michael Gaster,et al.  A critical assessment of the h‐index , 2012, BioEssays : news and reviews in molecular, cellular and developmental biology.

[71]  Anthony F. J. van Raan Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups , 2013, Scientometrics.

[72]  Kevin W. Boyack,et al.  Using global mapping to create more accurate document-level maps of research fields , 2011, J. Assoc. Inf. Sci. Technol..

[73]  Charles Oppenheim,et al.  Using the h-index to rank influential British researchers in information science and librarianship , 2007, J. Assoc. Inf. Sci. Technol..

[74]  Lutz Bornmann,et al.  What do we know about the h index? , 2007, J. Assoc. Inf. Sci. Technol..

[75]  D J PRICE,et al.  NETWORKS OF SCIENTIFIC PAPERS. , 1965, Science.

[76]  Jerome K. Vanclay,et al.  On the robustness of the h-index , 2007, J. Assoc. Inf. Sci. Technol..

[77]  Kevin W. Boyack,et al.  Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately? , 2010, J. Assoc. Inf. Sci. Technol..

[78]  Henry G. Small,et al.  Clustering thescience citation index® using co-citations , 1985, Scientometrics.

[79]  Chun-Ting Zhang,et al.  The e-Index, Complementing the h-Index for Excess Citations , 2009, PloS one.

[80]  Jean Anderson Eloy,et al.  Is NIH funding predictive of greater research productivity and impact among academic otolaryngologists? , 2013, The Laryngoscope.

[81]  Richard S. J. Tol,et al.  A rational, successive g-index applied to economics departments in Ireland , 2008, J. Informetrics.

[82]  E. Garfield The history and meaning of the journal impact factor. , 2006, JAMA.

[83]  Vincent Larivière,et al.  Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature , 2013, J. Assoc. Inf. Sci. Technol..

[84]  Ushma S. Neill,et al.  Publish or perish, but at what cost? , 2008, The Journal of clinical investigation.

[85]  Alan Fersht,et al.  The most influential journals: Impact Factor and Eigenfactor , 2009, Proceedings of the National Academy of Sciences.

[86]  Linda Butler,et al.  Using a balanced approach to bibliometrics: quantitative performance measures in the Australian Research Quality Framework , 2008 .

[87]  Loet Leydesdorff,et al.  Can scientific journals be classified in terms of aggregated journal-journal citation relations using the Journal Citation Reports? , 2009, J. Assoc. Inf. Sci. Technol..

[88]  Péter Jacsó,et al.  Google Scholar Author Citation Tracker: is it too little, too late? , 2012 .

[89]  Henry G. Small,et al.  Clustering the science citation index using co-citations. II. Mapping science , 1985, Scientometrics.

[90]  Ronald Rousseau,et al.  A proposal for a dynamic h-type index , 2008, J. Assoc. Inf. Sci. Technol..

[91]  Michael Schreiber,et al.  An empirical investigation of the g-index for 26 physicists in comparison with the h-index, the A-index, and the R-index , 2008, J. Assoc. Inf. Sci. Technol..

[92]  Francisco Herrera,et al.  hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices , 2010, Scientometrics.

[93]  Nicolás Robinson-García,et al.  The Google scholar experiment: How to index false papers and manipulate bibliometric indicators , 2013, J. Assoc. Inf. Sci. Technol..

[94]  Roberto Verganti,et al.  Evaluating the performance of academic departments: an analysis of research-related output efficiency , 2012 .

[95]  Ronald Rousseau,et al.  Probing the h-core: an investigation of the tail–core ratio for rank distributions , 2010, Scientometrics.

[96]  Richard A. Posner,et al.  The Theory and Practice of Citations Analysis, with Special Reference to Law and Economics , 1999 .

[97]  Lutz Bornmann,et al.  Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine , 2008, J. Assoc. Inf. Sci. Technol..

[98]  M. Vasei,et al.  Normalized impact factor (NIF): An adjusted method for calculating the citation rate of biomedical journals , 2011, J. Biomed. Informatics.

[99]  Barry Bozeman,et al.  Academic careers, patents, and productivity: industry experience as scientific and technical human capital , 2005 .

[100]  Qiang Wu The w-index: A significant improvement of the h-index , 2008 .

[101]  Mary Frank Fox,et al.  Social-Organizational Characteristics of Work and Publication Productivity among Academic Scientists in Doctoral-Granting Departments , 2007 .

[102]  Adam Eyre-Walker,et al.  The Assessment of Science: The Relative Merits of Post-Publication Review, the Impact Factor, and the Number of Citations , 2013, PLoS biology.

[103]  Steffen Staab,et al.  Web Science , 2013, Informatik-Spektrum.

[104]  David Wallace,et al.  Towards a Tool for Characterizing the Progression of Academic Research , 2011 .

[105]  Mônica G. Campiteli,et al.  Is it possible to compare researchers with different scientific interests? , 2006, Scientometrics.

[106]  Mark Hahnel Exclusive: figshare a new open data project that wants to change the future of scholarly publishing , 2012 .