Misleading Rankings of Research in Business: A Reply

Sönke Albers (2009) states in his abstract that he wants to critically evaluate our ranking of journals for economics and business; yet, that is not what he does. First and foremost, we do not develop a single ranking; instead, on the basis of four existing rankings, we construct four different meta-rankings through a methodologically sound imputation procedure, which is described in great detail in Schulze et al. (2008a, pp. 293–300). This imputation extends each base ranking to more than 2,800 journals while preserving the underlying logic of the list. It thus allows evaluating publications in journals that the original base ranking does not cover – and thus minimizes a problem that all journal lists had: they were too short! Frequently, research output could not be evaluated appropriately because a number of research outlets were not rated. Instead of imposing one list as the relevant one, we offer four different meta rankings based on two very different approaches – impact assessment and expert opinion approach – and we are very explicit about the relative advantages of both approaches, rather than criticizing just one. The user should choose the rankings knowing their relative strengths and weaknesses. Because we do not create new rankings, but impute on the basis of existing rankings, each of our imputed rankings will have the flaws and strengths of the underlying base ranking and the rankings used for imputation. Albers now claims that one of the existing rankings, i.e. the Ritzberger list and the imputed list based on Ritzberger, lacks ‘face validity’. His criterion is essentially that any list should coincide very highly with the lists that he thinks are relevant (see his Table 4). He does not make any attempt to reflect critically on the weaknesses of these lists, and thus while he is very critical towards one list he is very uncritical towards others. In the light of different weaknesses of all rankings a balanced view would take the relative strengths and weaknesses into account rather than just postulating a group of lists as