Can we do better than existing author citation metrics?

Throughout the world, research bodies have begun to quantify research quality through citation analysis. Both Britain [1] and Australia [2] now incorporate a bibliometric element when assessing institutional research output. The Institute for Research Information and Quality Assurance was recently established with the core goal of evaluating research performance funded by the German Research Foundation. In China [3], authors have been asked to publish only in journals indexed in the Institute for Scientific Information (ISI) Science Citation Index and which therefore receive an Impact Factor. Some individual institutions have even more specific requirements [4]. With so many researchers and specialised disciplines, it is tempting for funding agencies and governments to reduce an author’s publication career to a single metric. It is also spectacularly difficult [5]. When attempting to follow Professor Hopper’s advice to take one measurement over a thousand expert opinions, the challenge is in making that one measurement accurate. To properly measure author impact, such a metric would need to have a number of important attributes, many of them involving some complex statistics [6]. We may consider the following a simplified list of some of the most crucial criteria. First, the metric would have to be unambiguous, so that two calculations of the metric could not reach different results and so that only data for the author under study are included. Second, it would need to fairly compare authors from different subjects or countries and publishing different types of papers. Third, it would need to take account of time – both the age of the articles and the length of the author’s publication career. Fourth, the metric must be easily calculated, particularly if it is to be systematically generated for large numbers of authors. Current methods of citation analysis based on this system fail our second criterion, because case studies, practitioner and clinical articles receive a lower level of citation that does not reflect their actual usefulness to the academic community [7]. Additionally, our two citation indices, Web of Science and Scopus, both index a higher proportion of English-language journals than foreign language, skewing results in favour of the West and towards the US and UK in particular [8]. But the problem is with more than the data: if we look at the author metrics currently in use, they are found to be lacking in other respects.