Ranking journals in business and management: a statistical analysis of the Harzing data set

Creating rankings of academic journals is an important but contentious issue. It is of especial interest in the U.K. at this time (2007) as we are only one year away from getting the results of the next Research Assessment Exercise (RAE) the importance of which, for U.K. universities, can hardly be overstated. The purpose of this paper is to present a journal ranking for business and management based on a statistical analysis of the Harzing data set which contains 13 rankings. The primary aim of the analysis is two-fold – to investigate relationships between the different rankings, including that between peer rankings and citation behaviour; and to develop a ranking based on four groups that could be useful for the RAE. Looking at the different rankings, the main conclusions are that there is in general a high degree of conformity between them as shown by a principal components analysis. Cluster analysis is used to create four groups of journals relevant to the RAE. The higher groups are found to correspond well with previous studies of top management journals and also gave, unlike them, equal coverage to all the management disciplines. The RAE Business and Management panel have a huge and unenviable task in trying to judge the quality of over 10,000 publications and they will inevitably have to resort to some standard mechanistic procedures to do so. This work will hopefully contribute by producing a ranking based on a statistical analysis of a variety of measures.

[1]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[2]  Paul Benjamin Lowry,et al.  Global Journal Prestige and Supporting Disciplines: A Scientometric Study of Information Systems Journals , 2004, J. Assoc. Inf. Syst..

[3]  J. Gower.  A General Coefficient of Similarity and Some of Its Properties , 1971 .

[4]  Alan Agresti,et al.  Mathematical and computer modelling reports: A model for agreement between ratings on an ordinal scale , 1988 .

[5]  Charles Jennings,et al.  Citation Data: The Wrong Impact? * * Reprinted with permission from Nature Neuroscience, 1, December, 1998, 641-643. , 2001, Cortex.

[6]  Jae-On Kim,et al.  Multivariate Analysis of Ordinal Variables , 1975, American Journal of Sociology.

[7]  R. A. Rees,et al.  On “The Social Responsibility of Operational Research” , 1976 .

[8]  Brian Everitt,et al.  Cluster analysis , 1974 .

[9]  Laurie McAulay,et al.  Citation as effortful voting: A reply , 1996 .

[10]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[11]  Reform of higher education research assessment and funding , 2006 .

[12]  Sanford Labovitz,et al.  The Assignment of Numbers to Rank Order Categories , 1970 .

[13]  John R. Doyle,et al.  Judging the quality of research in business schools: The UK as a case study , 1995 .

[14]  Henk F. Moed,et al.  Journal impact measures in bibliometric research , 2004, Scientometrics.

[15]  Sanford Labovitz,et al.  Some Observations on Measurement and Statistics , 1967 .

[16]  F. DuBois,et al.  Ranking the International Business Journals , 2000 .

[17]  Quentin L. Burrell,et al.  Modeling citation behavior in Management Science journals , 2006, Inf. Process. Manag..

[18]  Mark D. Miller,et al.  Examining differences across journal rankings , 2005, CACM.

[19]  M. Aldenderfer.  Cluster Analysis , 1984 .

[20]  Xianggui Qu,et al.  Multivariate Data Analysis , 2007, Technometrics.

[21]  Michael John Jones,et al.  Journal evaluation methodologies: A balanced response , 1996 .

[22]  Bruno D. Zumbo,et al.  Is the selection of statistical methods governed by level of measurement , 1993 .

[23]  Michael Rowlinson,et al.  Journal Rankings in Business and Management and the 2001 Research Assessment Exercise in the UK , 2004 .

[24]  A. Tahai,et al.  A revealed preference study of management journals' direct influences , 1999 .

[25]  Rajiv Kohli,et al.  A multiple criteria assessment of decision technology system journal quality , 2001, Inf. Manag..

[26]  John Mingers,et al.  Exploring the dynamics of journal citations: Modelling with s-curves , 2008, J. Oper. Res. Soc..

[27]  K. Peffers,et al.  Identifying and Evaluating the Universe of Outlets for Information Systems Research: Ranking the Journals , 2003 .

[28]  Charles Baden-Fuller,et al.  Making and Measuring Reputations: The Research Ranking of European Business Schools , 2000 .

[29]  Michael R. Anderberg,et al.  Cluster Analysis for Applications , 1973 .

[30]  Alan C.B. Tse,et al.  Using mathematical programming to solve large ranking problems , 2001, J. Oper. Res. Soc..

[31]  A. Liebetrau.  Measures of association , 1983 .

[32]  Alexander von Eye,et al.  Models for Ordinal Agreement Data , 2001 .

[33]  Pairin Katerattanakul,et al.  Are European IS Journals under-rated? An answer based on citation analysis , 2003, Eur. J. Inf. Syst..

[34]  I Horowitz.  Preference-neutral attribute weights in the journal-ranking problem , 2003, J. Oper. Res. Soc..

[35]  J. Gibbons.  Nonparametric measures of association , 1993 .

[36]  G. Easton,et al.  Marketing Journals and the Research Assessment Exercise , 2003 .

[37]  Maurice Pendlebury,et al.  Judging the quality of research in business schools: A comment from accounting , 1996 .

[38]  Laurie McAulay,et al.  The judge, the model of the judge, and the model of the judged as judge: Analyses of the UK 1992 research assessment exercise data for business and management studies , 1996 .