The race for the impact factor

Every year during the second half of June, journal editors and publishers become nervous and jittery, and perhaps even sleepless, while waiting impatiently for the publication of their journal s impact factors (IFs) for the year by Thomson Reuters (formerly Thomson Scientific). In recent years, the IF has gained a mythical influence on the prestige of journals and their published papers which has resulted in a race for the impact factor . Editors try to increase their journals IFs as much as possible and authors try to publish their papers in journals with the highest-possible IFs. This editorial was written a short time after publication of the 2008 IFs and the rush of communications by editors and publishers that followed it. The term impact factor was introduced in 1963 by Garfield and Sher (1963) for the purpose of improving the management of library journals collections. The idea was to devise an instrument that would take into consideration the importance of a journal, regardless of the number of papers it publishes. Sorting journals by IF allowed for the inclusion of many small but influential journals in bibliographic collections. Over the years, the term IF evolved gradually as a surrogate of both journal and author impact and influence, so that the mere acceptance of a paper by a high IF journal is perceived as recognition of the paper s importance. As such, it is used increasingly by academic committees, funding agencies and institutional evaluation panels in their deliberations and decisions about academic promotion and grant funding. Indeed, in some instances government funding of universities is influenced by IF considerations. The IF for a given year is calculated by dividing the total number of citations of journal papers published during a 2year period before that year by the total number of citable items published during the same 2-year period. The IF of Journal of Sleep Research (JSR) for 2008 was 3.255, as during 2006–2007 JSR published 110 citable papers that were cited 358 times in 2008. The IF of the other leading sleep journals were 4.475 for Sleep (335 papers cited 1499 times), 3.163 for Sleep Medicine (203 papers cited 642 times) and 6.143 for Sleep Medicine Reviews, that publish exclusively reviews (63 papers cited for 387 times). The growing reliance on IF for evaluating scholarly achievements has led to mounting criticism of the index and its indiscriminate use. One of the most widespread criticisms is that the distribution of citations of papers within a journal is skewed heavily and therefore cannot be represented adequately by the statistical average. Analysis of the citations distribution in the sleep journals will demonstrate this point. Forty-two of the 110 (38.1%) citable papers published during 2006–2007 in JSR were not cited or had a single citation in 2008, 24 (21.8%) were cited at least five times and five (4.5%) were cited at least 10 times (Bjorvaten et al., 2007; Fogel and Smith, 2006; Killgore et al., 2006; Russo et al., 2007; Tucker et al., 2007). Similarly, for the journal Sleep, 114 (34%) papers published in 2006–2007 were not cited in 2008 or cited only once, and 27 (8.05%) were cited at least 10 times. For Sleep Medicine Reviews, 14 (22.2%) were not cited or cited only once, 10 were cited at least 10 times (15.8%) and two (2.7%) were cited at least 20 times. It is noteworthy that a single paper published in 2006 and cited 64 times in 2008 accounted for 19.1% of Sleep Medicine Reviews IF for that year. Thus, a journal s IFs are determined mainly by a limited number of papers that are highly cited in the first 2 years after publication. Notably, even a single highly cited paper can have a disproportionately large effect on a journal s IF. Criticism has also focused on the fact that, as currently calculated, IF is biased heavily by factors irrelevant to the scientific influence of the papers. The IF is related directly to the area of research. Journals of specialized areas of science will never attain the IF of more general scientific journals. Furthermore, journals of fields in which the average number of citations per paper is traditionally low, such as mathematics, will never attain the IF of journals of fields in which the number of citations is high, such as biology or medicine. In addition, the fact that IF is based on a measurement unit of 2 years favours rapidly changing disciplines and hot subjects, and journals with a short time-lag between submission and publication. Accordingly, the three JSR papers published in 2006–2007 that had the highest number of citations in 2008 were on hot subjects of sleep research: the effect of sleep on learning, the relation of sleep duration to metabolism and the effects of sleep deprivation. A particularly troublesome problem is that the IF can be manipulated and engineered. As the IF is calculated by a simple formula comprising only two numbers – one, total citations and the second, total citable items – increasing the nominator and ⁄or decreasing the denominator can boost the IF. Editorial policies can inflate the nominator by bluntly encouraging authors to include citations from the same journal instead of citations from other journals. Although such behaviour has been discouraged by the academic and scientific community, the number of self-citations vary greatly between journals and it is not unusual to find 18–20% self citations, which adds considerably to the journal s IF. The most recent data on self-citations for the sleep journals are: JSR 5%, Sleep Medicine Reviews 4%, Sleep 10% and Sleep Medicine 8%. Impact factors can be manipulated in more subtle ways. Editorial policies to publish summaries of past publications in the journal such as A year in review -type papers, to favour papers that seem potentially highly citable regardless of their scientific quality, and to publish a large proportion of review papers over the less-cited research papers will increase the nominator of the IF formula considerably. The difference in J. Sleep Res. (2009) 18, 283–284 Editorial