The seven deadly sins of world university ranking: a summary from several papers

ABSTRACT World university rankings use the weight-and-sum approach to process data. Although this seems to pass the common sense test, it has statistical problems. In recent years, seven such problems have been uncovered: spurious precision, weight discrepancies, assumed mutual compensation, indictor redundancy, inter-system discrepancy, negligence of indicator scores and inconsistency between changes in ranking and ‘Overall’. These issues render the validity of ranking results suspect. Moreover, discussion on problems in world university ranking has been conducted mostly at the verbal level with little substantiation of supporting statistics. It is argued that discussion on ranking issues needs statistical evidential support. Modifications to the approach of processing ranking data are suggested.

[1]  Kaycheng Soh,et al.  What the Overall doesn’t tell about world university rankings: examples from ARWU, QSWUR, and THEWUR in 2013 , 2015 .

[2]  Nicholas P Leveillee.  Copernicus, Galileo, and the Church: Science in a Religious World , 2011 .

[3]  Kaycheng Soh,et al.  Misleading university rankings: cause and cure for discrepancies between nominal and attained weights , 2013 .

[4]  S. K. Cheng.  The U21 rankings of National Higher Education Systems: re-analyze to optimize , 2012 .

[5]  Ross Williams,et al.  U21 Ranking of National Higher Education Systems 2015 , 2012 .

[6]  K. Soh.  Problems of Indicator Weights and Multicolinearity in World University Rankings: Comparisons of Three Systems. , 2014 .

[7]  M. Gunnarsson.  QS World University Rankings , 2011 .

[8]  K. Soh.  Rectifying an honest error in world university rankings: a solution to the problem of indicator weight discrepancies , 2013 .

[9]  Nian Cai Liu,et al.  The Academic Ranking of World Universities. , 2005 .

[11]  K. Soh,et al.  Don’t read university rankings like reading football league tables: Taking a close look at the indicators , 2011 .

[12]  Soh Kaycheng,et al.  Multicolinearity and Indicator Redundancy Problem in World University Rankings: An Example Using Times Higher Education World University Ranking 2013–2014 Data , 2015 .

[13]  L. Ong,et al.  The Big Mac Index , 2003 .

[14]  Wg.Cdr. Pongphet Congpuong.  How to lie With Statistics , 2013 .

[15]  K. Soh.  Nearing world-class: Singapore’s two universities in QSWUR 2015/16 , 2016 .

[16]  K. Soh,et al.  A tale of two cities’ university rankings: comparing Hong Kong and Singapore , 2014 .

[17]  D. Ross,et al.  Times Higher Education World University Rankings , 2017 .

[18]  Soh Kay Cheng,et al.  U21 Ranking of National Higher Education Systems: a project sponsored by Universitas 21 , 2012 .

[19]  K. Soh.  Nominal versus attained weights in Universitas 21 Ranking , 2014 .

[20]  World University Rankings: What Is in for Top Ten East Asian Universities?. , 2012 .

[21]  Kay Cheng Soh,et al.  Multicolinearity and indicator redundancy problem in world university rankings: An example using THEWUR 2013-2014 data , 2015 .

[22]  Times Higher Education 100 under 50 ranking: old wine in a new bottle? , 2013 .