R-Score: Reputation-based Scoring of Research Groups

To manage the problem of having a higher demand for resources than availability of funds, research funding agencies usually rank the major research groups in their area of knowledge. This ranking relies on a careful analysis of the research groups in terms of their size, number of PhDs graduated, research results and their impact, among other variables. While research results are not the only variable to consider, they are frequently given special attention because of the notoriety they confer to the researchers and the programs they are affiliated with. In here we introduce a new metric for quantifying publication output, called R-Score for reputation-based score, which can be used in support to the ranking of research groups or programs. The novelty is that the metric depends solely on the listings of the publications of the members of a group, with no dependency on citation counts. R-Score has some interesting properties: (a) it does not require access to the contents of published material, (b) it can be curated to produce highly accurate results, and (c) it can be naturally used to compare publication output of research groups (e.g., graduate programs) inside a same country, geographical area, or across the world. An experiment comparing the publication output of 25 CS graduate programs from Brazil suggests that R-Score can be quite useful for providing early insights into the publication patterns of the various research groups one wants to compare.

[1]  Johan Bollen,et al.  Toward alternative metrics of journal impact: A comparison of download and citation data , 2005, Inf. Process. Manag..

[2]  Michael Ley,et al.  The DBLP Computer Science Bibliography: Evolution, Research Issues, Perspectives , 2002, SPIRE.

[3]  Andreas Thor,et al.  Citation analysis of database publications , 2005, SGMD.

[4]  Carlos José Pereira de Lucena,et al.  Assessing the research and education quality of the top Brazilian Computer Science graduate programs , 2008, SGCS.

[5]  Carl D. Meyer,et al.  Stochastic Complementation, Uncoupling Markov Chains, and the Theory of Nearly Reducible Systems , 1989, SIAM Rev..

[6]  Dongwon Lee,et al.  Toward alternative measures for ranking venues: a case of database research community , 2007, JCDL '07.

[7]  Riyaz Sikora,et al.  Assessing the relative influence of journals in a citation network , 2005, CACM.

[8]  E. Garfield,et al.  Citation indexes for science. , 1956, Science.

[9]  Jian Pei,et al.  Detecting topic evolution in scientific literature: how can citations help? , 2009, CIKM.

[10]  Winfried K. Grassmann,et al.  Regenerative Analysis and Steady State Distributions for Markov Chains , 1985, Oper. Res..

[11]  J. Nicolaisen Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics , 2010 .

[12]  Amy Nicole Langville,et al.  Google's PageRank and beyond - the science of search engine rankings , 2006 .

[13]  C. Spearman The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[14]  Soongoo Hong,et al.  Objective quality ranking of computing journals , 2003, CACM.

[15]  Dongwon Lee,et al.  Measuring conference quality by mining program committee characteristics , 2007, JCDL '07.

[16]  Johan Bollen,et al.  Journal status , 2006, Scientometrics.

[17]  Mark Newman,et al.  Networks: An Introduction , 2010 .

[18]  Gideon S. Mann,et al.  Bibliometric impact measures leveraging topic analysis , 2006, Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06).

[19]  Judit Bar-Ilan,et al.  Which h-index? — A comparison of WoS, Scopus and Google Scholar , 2008, Scientometrics.

[20]  Marilyn Bohl,et al.  Information processing , 1971 .

[21]  D. Christakis,et al.  Impact factor: a valid measure of journal quality? , 2003, Journal of the Medical Library Association : JMLA.

[22]  Peter Ingwersen,et al.  Using citations for ranking in digital libraries , 2006, Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06).

[23]  Thorsten Joachims,et al.  Information genealogy: uncovering the flow of ideas in non-hyperlinked document databases , 2007, KDD '07.

[24]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.