The weakening relationship between the impact factor and papers' citations in the digital age

Historically, papers have been physically bound to the journal in which they were published; but in the digital age papers are available individually, no longer tied to their respective journals. Hence, papers now can be read and cited based on their own merits, independently of the journal's physical availability, reputation, or impact factor (IF). We compare the strength of the relationship between journals' IFs and the actual citations received by their respective papers from 1902 to 2009. Throughout most of the 20th century, papers' citation rates were increasingly linked to their respective journals' IFs. However, since 1990, the advent of the digital age, the relation between IFs and paper citations has been weakening. This began first in physics, a field that was quick to make the transition into the electronic domain. Furthermore, since 1990 the overall proportion of highly cited papers coming from highly cited journals has been decreasing and, of these highly cited papers, the proportion not coming from highly cited journals has been increasing. Should this pattern continue, it might bring an end to the use of the IF as a way to evaluate the quality of journals, papers, and researchers. © 2012 Wiley Periodicals, Inc.

[1]  Jonas Lundberg,et al.  Lifting the crown - citation z-score , 2007, J. Informetrics.

[2]  Thed N. van Leeuwen,et al.  Improving the Accuracy of Institute for Scientific Informations's Journal Impact Factors , 1995, J. Am. Soc. Inf. Sci..

[3]  Ted Bergstrom Papers The Eigenfactor Metrics: A network approach to assessing scholarly journals , 2010 .

[4]  Harvey A. Averch,et al.  Exploring the cost-efficiency of basic research funding in chemistry , 1989 .

[5]  Emma Hill,et al.  Show me the data. , 1998 .

[6]  N. Mohaghegh,et al.  WHY THE IMPACT FACTOR OF JOURNALS SHOULD NOT BE USED FOR EVALUATING RESEARCH , 2005 .

[7]  D. Aksnes CHARACTERISTICS OF HIGHLY CITED PAPERS , 2003 .

[8]  George A. Lozano,et al.  A new criterion for allocating research funds: ‘impact per dollar’ , 2010 .

[9]  Vincent Larivière,et al.  History of the journal impact factor: Contingencies and consequences , 2009, Scientometrics.

[10]  Félix de Moya Anegón,et al.  The SJR indicator: A new indicator of journals' scientific prestige , 2009, ArXiv.

[11]  Henk F. Moed,et al.  Measuring contextual citation impact of scientific journals , 2009, J. Informetrics.

[12]  Richard Rothenberg,et al.  The impact factor follies. , 2008, Epidemiology.

[13]  Carl T. Bergstrom,et al.  The Eigenfactor MetricsTM: A Network Approach to Assessing Scholarly Journals , 2010, Coll. Res. Libr..

[14]  Endel Põder,et al.  Let's correct that small mistake , 2010, J. Assoc. Inf. Sci. Technol..

[15]  Glenn Ellison,et al.  Is Peer Review in Decline? , 2007 .

[16]  Vincent Larivière,et al.  The impact factor's Matthew Effect: A natural experiment in bibliometrics , 2009, J. Assoc. Inf. Sci. Technol..

[17]  Loet Leydesdorff,et al.  Integrated Impact Indicators (I3) compared with Impact Factors (IFs): An alternative research design with policy implications , 2011, J. Assoc. Inf. Sci. Technol..

[18]  G H Whitehouse,et al.  Citation rates and impact factors: should they matter? , 2001, The British journal of radiology.

[19]  Nimish Vakil The Journal Impact Factor: Judging a Book by Its Cover , 2005, The American Journal of Gastroenterology.

[20]  Aleksandar Janca,et al.  Revisiting the Journal Impact Factor , 2000 .