A comparison of 17 article-level bibliometric indicators of institutional research productivity: Evidence from the information management literature of China

Abstract Periodically ranking institutional research productivity is necessary not only to understand the status of the development of related fields but also to identify gaps and take appropriate corrective steps. Many bibliometric indicators contribute to the assessment of institutional research productivity, but the appropriateness of the indicator and the relationships between different indicators are topics that have not been addressed. For this reason, an indicator framework for the ranking of institutional research performance, which consists of a count of published articles and quality weighted dimensions, is developed. Based on the literature review, 17 indicators in the framework are chosen for study. Based on these indicators, experiments are conducted to rank Chinese institutions in the field of Information Management. Kendall's tau rank correlation coefficient (τ) is calculated, and all pairwise correlations are between 0.34 and 0.98. There are three primary findings: (1) among the article count indicators, the Straight count indicator is significantly different than others; (2) the rankings based on the indicators which are weighted by quality are consistent with those based on the indicators using article count; and (3) the Paper citation weighted indicator is sensitive to the procedure used for institutional ranking.

[1]  Ugo Finardi,et al.  Impact Factor and Citation Performance : n experimental study , 2013 .

[2]  Lorna Elizabeth Wildgaard,et al.  A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar , 2015, Scientometrics.

[3]  Mike W. Peng,et al.  Most cited articles and authors in global strategy research , 2006 .

[4]  Simon Hussain,et al.  Journal list fetishism and the ‘sign of 4’ in the ABS guide: A question of trust? , 2015 .

[5]  Dean Hendrix,et al.  An analysis of bibliometric indicators, National Institutes of Health funding, and faculty size at Association of American Medical Colleges medical schools, 1997-2007. , 2008, Journal of the Medical Library Association : JMLA.

[6]  Filippo Menczer,et al.  Quality versus quantity in scientific impact , 2014, J. Informetrics.

[7]  Yoris A. Au,et al.  Assessing Researcher Publication Productivity in the Leading Information Systems Journals: A 2005-2009 Update , 2011, Commun. Assoc. Inf. Syst..

[8]  Jean-François Molinari,et al.  A new methodology for ranking scientific institutions , 2008, Scientometrics.

[9]  Chokri Dridi,et al.  Ranking of Research Output of Agricultural Economics Departments in Canada and Selected U.S. Universities , 2010 .

[10]  REITs and Market Friction , 2014 .

[11]  Manoj K. Malhotra,et al.  Institutional research productivity in production and operations management , 1996 .

[12]  Ricky Y. K. Chan,et al.  Publications in Major Marketing Journals: An Analysis of Research Productivity of Asia-Pacific Universities , 2003 .

[13]  Kam C. Chan,et al.  A long-term assessment of finance research performance among Asia–Pacific academic institutions (1990–2008) , 2011 .

[14]  Yuh-Shan Ho,et al.  Comparison of independent research of China’s top universities using bibliometric indicators , 2012, Scientometrics.

[15]  J. Tata,et al.  An empirical assessment of institutional and individual research productivity in international operations management , 2000 .

[16]  Jianping Li,et al.  Ranking the research productivity of business and management institutions in Asia–Pacific region: empirical research in leading ABS journals , 2015, Scientometrics.

[17]  C. Harvey,et al.  Journal rankings and the ABS Journal Quality Guide , 2009 .

[18]  Jack Shih-Chieh Hsu,et al.  An Evaluation of Publication Productivity in Information Systems: 1999 to 2003 , 2005, Commun. Assoc. Inf. Syst..

[19]  Influential Journals, Institutions and Researchers in Real Estate , 2006 .

[20]  Heng Liu,et al.  A Ranking Analysis of the Management Schools in Greater China (2000–2010): Evidence From the SSCI Database , 2014 .

[21]  Xiaohua Yu,et al.  An updated ranking of the economic research institutions in China (2000-2009) , 2010 .

[22]  Mike W. Peng,et al.  Research rankings of Asia Pacific business schools: Global versus local knowledge strategies , 2008 .

[23]  Gurpreet Dhillon,et al.  Challenging Assumptions for Strategic Information Systems Planning: Theoretical Perspectives , 2000, Commun. Assoc. Inf. Syst..

[24]  Hung-Gay Fung,et al.  International business research: Trends and school rankings , 2006 .

[25]  Robert J. Kauffman,et al.  50th Anniversary Article: The Evolution of Research on Information Systems: A Fiftieth-Year Survey of the Literature in Management Science , 2004, Manag. Sci..

[26]  Jean-François Molinari,et al.  Mathematical aspects of a new criterion for ranking scientific institutions based on the h-index , 2008, Scientometrics.

[27]  L. Egghe.  An improvement of the h-index: the g-index , 2006 .

[28]  Steven H. Seggie,et al.  Prolific authors and institutions in leading international business journals , 2008 .

[29]  Kam C. Chan,et al.  Ranking Research Productivity in Accounting for Asia-Pacific Universities , 2005 .

[30]  Henk F. Moed,et al.  Multidimensional assessment of scholarly research impact , 2014, J. Assoc. Inf. Sci. Technol..

[31]  Jang C. Jin,et al.  East Asian rankings of economics departments , 2008 .

[32]  R. Rousseau,et al.  The R- and AR-indices: Complementing the h-index , 2007 .

[33]  T. Gamblin,et al.  Measuring the surgical academic output of an institution: the "institutional" H-index. , 2012, Journal of surgical education.

[34]  Quentin L. Burrell,et al.  Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data , 2007, Scientometrics.

[35]  Jung P. Shim,et al.  An examination of articles in the eight leading management information systems journals: 1980–1988 , 1991 .

[36]  Liming Liang,et al.  h-index sequence and h-index matrix: Constructions and applications , 2006, Scientometrics.

[37]  Pao-Long Chang,et al.  Bibliometric Overview of Operations Research/Management Science Research in Asia , 2008, Asia Pac. J. Oper. Res..

[38]  Joshua C. Hall,et al.  An ordinal ranking of economic institutions , 2016 .

[39]  Juan E. Iglesias,et al.  Scaling the h-index for different scientific ISI fields , 2006, Scientometrics.

[40]  T. Coupé.  REVEALED PERFORMANCES: WORLDWIDE RANKINGS OF ECONOMISTS AND ECONOMICS DEPARTMENTS, 1990–2000 , 2003 .

[41]  Detmar W. Straub,et al.  Measuring Researcher-Production in Information Systems , 2002, J. Assoc. Inf. Syst..

[42]  Somnath Lahiri,et al.  Ranking International Business Institutions and Faculty Members Using Research Publication as the Measure , 2012 .

[43]  Varun Grover,et al.  The Information Systems Field: Making a Case for Maturity and Contribution , 2012, J. Assoc. Inf. Syst..

[44]  Somnath Lahiri,et al.  India-focused publications in leading international business journals , 2011 .

[45]  Paul Benjamin Lowry,et al.  Global Journal Prestige and Supporting Disciplines: A Scientometric Study of Information Systems Journals , 2004, J. Assoc. Inf. Syst..

[46]  D. Griffith,et al.  Emerging themes in international business research , 2008 .

[47]  Yi Zhao,et al.  Bringing PageRank to the citation analysis , 2008, Inf. Process. Manag..

[48]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[49]  Declan Butler,et al.  University rankings smarten up , 2010, Nature.

[50]  John Mingers,et al.  Evaluating a department's research: Testing the Leiden methodology in business and management , 2013, Inf. Process. Manag..

[51]  Kam C. Chan,et al.  A quality-based global assessment of financial research , 2016 .

[52]  Carl R. Chen,et al.  Research productivity of the finance profession in the Asia-Pacific region , 2001 .

[53]  Susan Athey,et al.  An Evaluation of Research Productivity in Academic IT , 2000, Commun. Assoc. Inf. Syst..