Usage Impact Factor: the effects of sample characteristics on usage-based impact metrics

There exist ample demonstrations that indicators of scholarly impact analogous to the citation-based ISI Impact Factor can be derived from usage data. However, contrary to the ISI IF which is based on citation data generated by the global community of scholarly authors, so far usage can only be practically recorded at a local level leading to community-specific assessments of scholarly impact that are difficult to generalize to the global scholarly community. We define a journal Usage Impact Factor which mimics the definition of the Thomson Scientific's ISI Impact Factor. Usage Impact Factor rankings are calculated on the basis of a large-scale usage data set recorded for the California State University system from 2003 to 2005. The resulting journal rankings are then compared to Thomson Scientific's ISI Impact Factor which is used as a baseline indicator of general impact. Our results indicate that impact as derived from California State University usage reflects the particular scientific and demographic characteristics of its communities.

[1]  Johan Bollen,et al.  An architecture for the aggregation and analysis of scholarly usage data , 2006, Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06).

[2]  Thomas V Perneger,et al.  Competing interests: None declared. Ethical approval: Ethics committee of Côte d’Ivoire’s Ministry of Public Health and the Institutional Review Board of the US Centers for Disease Control and Prevention , 2004 .

[3]  Stevan Harnad,et al.  Earlier Web Usage Statistics as Predictors of Later Citation Impact , 2005, J. Assoc. Inf. Sci. Technol..

[4]  Johan Bollen,et al.  Mapping the structure of science through usage , 2006, Scientometrics.

[5]  J. Benichou,et al.  Reading factor: a new bibliometric criterion for managing digital libraries. , 2002, Journal of the Medical Library Association : JMLA.

[6]  Peter T. Shepherd.  The feasibility of developing and implementing journal usage factors: a research project sponsored by UKSG , 2007 .

[7]  Ian Rowlands,et al.  Revisiting 'obsolescence' and journal article 'decay' through usage data: an analysis of digital journal use by year of publication , 2005, Inf. Process. Manag..

[8]  Carol Tenopir,et al.  Measuring Total Reading of Journal Articles , 2006, D Lib Mag..

[9]  Herbert Van de Sompel,et al.  Reference Linking in a Hybrid Library Environment, Part 1: Frameworks for Linking , 1999, D Lib Mag..

[10]  Herbert Van de Sompel,et al.  Reference Linking in a Hybrid Library Environment, Part 2: SFX, a Generic Linking Solution , 1999, D Lib Mag..

[11]  Philip M. Davis,et al.  eJournal interface can influence usage statistics: Implications for libraries, publishers, and Project COUNTER , 2006, J. Assoc. Inf. Sci. Technol..

[12]  E. Garfield.  Journal impact factor: a brief review. , 1999, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[13]  Allen Kent,et al.  Use of a University Library Collection; A Progress Report on a Pittsburgh Study. , 1977 .

[14]  Johan Bollen,et al.  Evaluation of Digital Library Impact and User Communities by Analysis of Usage Patterns , 2002, D Lib Mag..

[15]  Johan Bollen,et al.  Detecting Research Trends in Digital Library Readership , 2003, ECDL.

[16]  J. Reedijk.  Sense and nonsense of science citation analyses: comments on the monopoly position of ISI and citation inaccuracies. Risks of possible misuse and biased citation and impact data. , 1998 .

[17]  Thed N. van Leeuwen,et al.  Citation delay in interdisciplinary knowledge exchange , 2001, Scientometrics.

[18]  Pauline A. Scales.  CITATION ANALYSES AS INDICATORS OF THE USE OF SERIALS: A COMPARISON OF RANKED TITLE LISTS PRODUCED BY CITATION COUNTING AND FROM USE DATA , 1976 .

[19]  The Impact Factor Game , 2006, PLoS medicine.

[20]  Kathleen Bauer,et al.  Evidence-based librarianship: Utilizing data from all available sources to make judicious print cancellation decisions , 2005 .

[21]  Stephen S. Murray,et al.  The bibliometric properties of article readership information , 2005, J. Assoc. Inf. Sci. Technol..

[22]  Henk F. Moed,et al.  Statistical relationships between downloads and citations at the level of individual documents within a single journal , 2005, J. Assoc. Inf. Sci. Technol..

[23]  Eugene Garfield,et al.  Citation indexing - its theory and application in science, technology, and humanities , 1979 .

[24]  Michael H. MacRoberts,et al.  Problems of citation analysis: A critical review , 1989, JASIS.

[25]  Brinley Franklin.  Library usage patterns in the electronic information environment , 2004, Inf. Res..

[26]  D. Christakis,et al.  Impact factor: a valid measure of journal quality? , 2003, Journal of the Medical Library Association : JMLA.

[27]  Judy Luther.  White Paper on Electronic Journal Usage Statistics , 2002 .

[28]  M. Tsay.  The relationship between journal use in a medical library and citation use. , 1998, Bulletin of the Medical Library Association.

[29]  Stephen S. Murray,et al.  Worldwide Use and Impact of the NASA Astrophysics Data System Digital Library , 2009, J. Assoc. Inf. Sci. Technol..

[30]  Henk F. Moed,et al.  Publication delays in the science field and their relationship to the ageing of scientific literature , 2006, Scientometrics.

[31]  Herbert Van de Sompel,et al.  Reference Linking in a Hybrid Library Environment , 1999 .

[32]  T. Opthof,et al.  Sense and nonsense about the impact factor. , 1997, Cardiovascular research.

[33]  Peter Weingart,et al.  Impact of bibliometrics upon the science system: Inadvertent consequences? , 2005, Scientometrics.

[34]  Daniel M. Dollar,et al.  Evidence-based librarianship: Utilizing data from all available sources to make judicious print cancellation decisions , 2005 .

[35]  Herbert Van de Sompel,et al.  Open Linking in the Scholarly Information Environment Using the OpenURL Framework , 2001, D Lib Mag..

[36]  S. D. De Groote,et al.  Online journals: impact on print journal usage. , 2001, Bulletin of the Medical Library Association.

[37]  Johan Bollen,et al.  Toward alternative metrics of journal impact: A comparison of download and citation data , 2005, Inf. Process. Manag..