Correlation between the Journal Impact Factor and three other journal citation indices

To determine the degree of correlation among journal citation indices that reflect the average number of citations per article, the most recent journal ratings were downloaded from the websites publishing four journal citation indices: the Institute of Scientific Information’s journal impact factor index, Eigenfactor’s article influence index, SCImago’s journal rank index and Scopus’ trend line index. Correlations were determined for each pair of indices, using ratings from all journals that could be identified as having been rated on both indices. Correlations between the six possible pairings of the four indices were tested with Spearman’s rho. Within each of the six possible pairings, the prevalence of identifiable errors was examined in a random selection of 10 journals and among the 10 most discordantly ranked journals on the two indices. The number of journals that could be matched within each pair of indices ranged from 1,857 to 6,508. Paired ratings for all journals showed strong to very strong correlations, with Spearman’s rho values ranging from 0.61 to 0.89, all p < 0.001. Identifiable errors were more common among scores for journals that had very discordant ranks on a pair of indices. These four journal citation indices were significantly correlated, providing evidence of convergent validity (i.e. they reflect the same underlying construct of average citability per article in a journal). Discordance in the ranking of a journal on two indices was in some cases due to an error in one index.

[1]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[2]  A. Melemeni,et al.  Academic anesthesiologists’ views on the importance of the impact factor of scientific journals: a North American and European survey , 2001, Canadian journal of anaesthesia = Journal canadien d'anesthesie.

[3]  Ronald Rousseau,et al.  The immediacy index and the journal impact factor: two highly correlated derived measures , 2004 .

[4]  M. Schein,et al.  What American surgeons read: a survey of a thousand Fellows of the American College of Surgeons. , 2000, Current surgery.

[5]  G. Davis,et al.  A comparison of Australian university output using journal impact factors , 2005, Scientometrics.

[6]  Ronald Rousseau.  On the relation between the WoS impact factor, the Eigenfactor, the SCImago Journal Rank, the Article Influence Score and the journal h-index , 2009 .

[7]  David Adam,et al.  Citation analysis: The counting house , 2002, Nature.

[8]  David Cyranoski,et al.  Cash for papers: putting a premium on publication , 2006, Nature.

[9]  Lars Iselid,et al.  Web of Science and Scopus: a journal title overlap study , 2008, Online Inf. Rev..

[10]  Hiroshi Itsumura,et al.  The relationships among the citation measures and the factors influence on them , 1984 .

[11]  Robert P. Dellavalle,et al.  Emerging alternatives to the impact factor , 2008, OCLC Syst. Serv..

[12]  Andrew P Kurmis,et al.  Understanding the limitations of the journal impact factor. , 2003, The Journal of bone and joint surgery. American volume.

[13]  Loet Leydesdorff.  How are new citation-based journal indicators adding to the bibliometric toolbox? , 2009 .

[14]  Chi-Kim Cheung,et al.  Audience Matters: A Study of How Authors Select Educational Journals , 2008 .

[15]  Philip M. Davis.  Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? , 2008 .

[16]  Ee Nkereuwem,et al.  Citation Ranking Versus Subjective Evaluation In The Determination Of Journal Hierarchies In Information Science , 2008 .

[17]  Arthur P. Young,et al.  Citation as a Form of Library Use , 1994 .

[18]  J. C. Smart,et al.  Consumption factor scores of psychology journals: Scientometric properties and qualitative implications , 1982, Scientometrics.

[19]  Johan Bollen,et al.  A Principal Component Analysis of 39 Scientific Impact Measures , 2009, PloS one.

[20]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[21]  Miguel A Hernán,et al.  Epidemiologists (of all people) should question journal impact factors. , 2008, Epidemiology.

[22]  N Wade,et al.  Citation analysis: a new tool for science administrators. , 1975, Science.

[23]  R. Monastersky.  The number that's devouring science , 2005 .

[24]  Liwen Vaughan,et al.  Can electronic journal usage data replace citation data as a measure of journal use? An empirical examination , 2006 .

[25]  Ricardo Arencibia-Jorge,et al.  Comparison of SCImago journal rank indicator with journal impact factor , 2008, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[26]  T. Opthof,et al.  Sense and nonsense about the impact factor. , 1997, Cardiovascular research.

[27]  Michael D. Gordon,et al.  Citation ranking versus subjective evaluation in the determination of journal hierachies in the social sciences , 1982, J. Am. Soc. Inf. Sci..

[28]  P. Cozby,et al.  Methods in behavioral research , 1977 .

[29]  A A Sandberg,et al.  The journal "impact factor": a misnamed, misleading, misused measure. , 1998, Cancer genetics and cytogenetics.

[30]  Massimo Franceschet,et al.  Journal influence factors , 2010, J. Informetrics.

[31]  Thed N. van Leeuwen,et al.  Characteristics of journal impact factors: The effects of uncitedness and citation distribution on the understanding of journal impact factors , 2005, Scientometrics.

[32]  Loet Leydesdorff,et al.  How are new citation-based journal indicators adding to the bibliometric toolbox? , 2009, J. Assoc. Inf. Sci. Technol..

[33]  Jesús Rey-Rocha,et al.  Some Misuses of Journal Impact Factor in Research Evaluation , 2001, Cortex.

[34]  J. Reedijk.  Sense and nonsense of science citation analyses: comments on the monopoly position of ISI and citation inaccuracies. Risks of possible misuse and biased citation and impact data. , 1998 .