A flexible approach for measuring author-level publishing performance

We propose a framework to evaluate, in relative terms, author-level publishing performance. To that end we introduce the publishing performance index (PPI) and the publishing performance box (PPB), and discuss the associated publishing profiles. We illustrate our approach conducting an extensive empirical application covering 472 top economists and developing several robustness tests. Instead of using a pre-designed measure without flexibility to adjust to the circumstances of each specific case, our approach accommodates alternative evaluation criteria, as defined by the evaluators. Beyond this key characteristic, our approach has some other important advantages: (1) it is easy to apply; (2) it is sensitive to the full list of publications and citations; (3) it is able to include additional dimensions of scientific performance beyond papers and citations; (4) it is a high granularity measure, providing a complete ranking of the authors under analysis.

[1]  R. Todeschini,et al.  Handbook of Bibliometric Indicators: Quantitative Tools for Studying and Evaluating Research , 2016 .

[2]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[3]  Carl T. Bergstrom,et al.  The Eigenfactor™ Metrics , 2008, The Journal of Neuroscience.

[4]  Qiang Wu,et al.  The w-index: A measure to assess scientific impact by focusing on widely cited papers , 2010, J. Assoc. Inf. Sci. Technol..

[5]  M. Schreiber.  A skeptical view on the Hirsch index and its predictive power , 2018, Physica Scripta.

[6]  Lutz Bornmann,et al.  Plots for visualizing paper impact and journal impact of single researchers in a single graph , 2017, Scientometrics.

[7]  Björn Hammarfelt,et al.  Indicators as judgment devices : An empirical study of citizen bibliometrics in research evaluation , 2017 .

[8]  Giovanni Abramo,et al.  How do you define and measure research productivity? , 2014, Scientometrics.

[9]  Dorte Henriksen,et al.  What factors are associated with increasing co-authorship in the social sciences? A case study of Danish Economics and Political Science , 2018, Scientometrics.

[10]  Xuan Zhen Liu,et al.  Modifying h-index by allocating credit of multi-authored papers whose author names rank based on contribution , 2012, J. Informetrics.

[11]  Joon-Oh Park,et al.  The Increasing Dominance of Teams in Production of Knowledge , 2011 .

[12]  Chun-Ting Zhang,et al.  The e-Index, Complementing the h-Index for Excess Citations , 2009, PloS one.

[13]  Francisco Herrera,et al.  h-Index: A review focused in its variants, computation and standardization for different scientific fields , 2009, J. Informetrics.

[14]  V. Larivière.  Global gender disparities in science , 2013 .

[15]  Birger Larsen,et al.  A review of the characteristics of 108 author-level bibliometric indicators , 2014, Scientometrics.

[16]  Andreas Walter,et al.  Do scholars in Economics and Finance react to alphabetical discrimination , 2015 .

[17]  D. Hamermesh.  Citations in Economics: Measurement, Uses and Impacts , 2015, SSRN Electronic Journal.

[18]  Thed N. van Leeuwen,et al.  Using Google Scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data , 2016 .

[19]  Ying Ding,et al.  Applying weighted PageRank to author citation networks , 2011, J. Assoc. Inf. Sci. Technol..

[20]  Sabrina Mayer,et al.  How does research productivity relate to gender? Analyzing gender differences for multiple publication dimensions , 2018, Scientometrics.

[21]  Olle Persson,et al.  The DCI index: Discounted cumulated impact-based research evaluation , 2008 .

[22]  Yuh-Shan Ho,et al.  The most frequently cited adsorption research articles in the Science Citation Index (Expanded). , 2012, Journal of colloid and interface science.

[23]  Philip Ball,et al.  Index aims for fair ranking of scientists , 2005, Nature.

[24]  Jorge E. Hirsch,et al.  hα: An index to quantify an individual’s scientific leadership , 2018, Scientometrics.

[25]  Kjell Hausken,et al.  The Ranking of Researchers by Publications and Citations , 2016 .

[26]  António Osório.  On the impossibility of a perfect counting method to allocate the credits of multi-authored publications , 2018, Scientometrics.

[27]  Zili Zhang,et al.  PR-Index: Using the h-Index and PageRank for Determining True Impact , 2016, PloS one.

[28]  Jirí Mazurek,et al.  A modification to Hirsch index allowing comparisons across different scientific fields , 2017, ArXiv.

[29]  Leo Egghe,et al.  The Hirsch index and related impact measures , 2010, Annu. Rev. Inf. Sci. Technol..

[30]  Philip J. Reny,et al.  How To Count Citations If You Must , 2016 .

[31]  Juan E. Iglesias,et al.  Scaling the h-index for different scientific ISI fields , 2006, Scientometrics.

[32]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics , 2015, Nature.

[33]  Lutz Bornmann,et al.  What are the top five journals in economics? A new meta-ranking , 2017 .

[34]  Jaime A. Teixeira da Silva,et al.  Multiple versions of the h-index: cautionary use for formal academic purposes , 2018, Scientometrics.

[35]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[36]  T. Fenner,et al.  A novel bibliometric index with a simple geometric interpretation , 2018, PloS one.

[37]  Tove Faber Frandsen,et al.  What is in a name? Credit assignment practices in different disciplines , 2010, J. Informetrics.

[38]  Ludo Waltman,et al.  F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations , 2014, J. Assoc. Inf. Sci. Technol..

[39]  Mike Thelwall,et al.  Google Scholar, Web of Science, and Scopus: a systematic comparison of citations in 252 subject categories , 2018, J. Informetrics.

[40]  Ali Daud,et al.  Indexing of authors according to their domain of expertise , 2017 .

[41]  Marek Kosmulski,et al.  Are you in top 1% (1‰)? , 2017, Scientometrics.

[42]  Koenraad Debackere,et al.  A concise review on the role of author self-citations in information science, bibliometrics and science policy , 2006, Scientometrics.

[43]  Francisco Herrera,et al.  hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices , 2010, Scientometrics.

[44]  Henk F. Moed,et al.  The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review , 2007 .

[45]  Helmut A. Abt.  A publication index that is independent of age , 2011, Scientometrics.

[46]  Lutz Bornmann,et al.  Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine , 2008, J. Assoc. Inf. Sci. Technol..

[47]  N. R. Dunnick,et al.  The H Index in Perspective. , 2017, Academic radiology.

[48]  Cassidy R. Sugimoto,et al.  Bibliometrics: Global gender disparities in science , 2013, Nature.

[49]  E. Leahey.  From Sole Investigator to Team Scientist: Trends in the Practice and Study of Research Collaboration , 2016 .

[50]  Mike Thelwall,et al.  Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management , 2015 .

[51]  Andreas Thor,et al.  The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute , 2016, J. Assoc. Inf. Sci. Technol..

[52]  Anthony F. J. van Raan.  Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups , 2013, Scientometrics.