There are increasing moves to deploy quantitative indicators in the assessment of research, particularly in the university sector. In Australia, discussions surrounding their use have long acknowledged the unsuitability of many standard quantitative measures for most humanities, arts, social science, and applied science disciplines. To fill this void, several projects are running concurrently. This paper details the methodology and initial results for one of the projects that aims to rank conferences into prestige tiers, and which is fast gaining a reputation for best practice in such exercises. The study involves a five-stage process: identifying conferences; constructing a preliminary ranking of these; engaging in extensive consultation; testing performance measures based on the rankings on ‘live’ data; and assessing the measures.In the past, many similar attempts to develop a ranking classification for publication outlets have faltered due to the inability of researchers to agree on a hierarchy. However the Australian experience suggests that when researchers are faced with the imposition of alternative metrics that are far less palatable, consensus is more readily achieved.
[1]
Linda Butler,et al.
Extending citation analysis to non-source items
,
2006,
Scientometrics.
[2]
Yannis Manolopoulos,et al.
A new perspective to automatically rank scientific conferences using digital libraries
,
2005,
Inf. Process. Manag..
[3]
L. Butler,et al.
Modifying publication practices in response to funding formulas
,
2003
.
[4]
Nian Cai Liu,et al.
The Academic Ranking of World Universities.
,
2005
.
[5]
Thed N. van Leeuwen,et al.
The application of bibliometric analyses in the evaluation of social science research. Who benefits from it, and why it is still feasible
,
2006,
Scientometrics.