A multidimensional analysis of Aslib proceedings - using everything but the impact factor

Purpose – The purpose of this paper is to show that the journal impact factor (IF) is not able to reflect the full impact of scholarly journals and provides an overview of alternative and complementary methods in journal evaluation. Design/methodology/approach – Aslib Proceedings (AP) is exemplarily analyzed with a set of indicators from five dimensions of journal evaluation, i.e. journal output, content, perception and usage, citations and management to accurately reflect its various strengths and weaknesses beyond the IF. Findings – AP has become more international in terms of authors and more diverse regarding its topics. Citation impact is generally low and, with the exception of a special issue on blogs, remains world average. However, an evaluation of downloads and Mendeley readers reveals that the journal is an important source of information for professionals and students and certain topics are frequently read but not cited. Research limitations/implications – The study is limited to one journal. ...

[1]  Vincent Larivière,et al.  Tweets vs. Mendeley readers: How do these two social media metrics differ? , 2014, it Inf. Technol..

[2]  Ludo Waltman,et al.  The relation between Eigenfactor, audience factor, and influence weight , 2010, J. Assoc. Inf. Sci. Technol..

[3]  Andrew M. Cox.  Flickr: a case study of Web2.0 , 2008, Aslib Proc..

[4]  Ian Rowlands,et al.  The attitudes and behaviours of illegal downloaders , 2010, Aslib Proc..

[5]  Ludo Waltman,et al.  Software survey: VOSviewer, a computer program for bibliometric mapping , 2009, Scientometrics.

[6]  Loet Leydesdorff,et al.  Normalization at the field level: fractional counting of citations , 2010, J. Informetrics.

[7]  Isabella Peters,et al.  Using social bookmarks and tags as alternative indicators of journal content description , 2012, First Monday.

[8]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[9]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[10]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[11]  Ian Rowlands,et al.  The missing link: journal usage metrics , 2007, Aslib Proc..

[12]  Ricardo Arencibia-Jorge,et al.  Comparison of SCImago journal rank indicator with journal impact factor , 2008, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[13]  Stefanie Haustein,et al.  Multidimensional journal evaluation , 2010 .

[14]  Michael Mabe,et al.  Dr Jekyll and Dr Hyde: author-reader asymmetries in scholarly publishing , 2002, Aslib Proc..

[15]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[16]  Vincent Larivière,et al.  History of the journal impact factor: Contingencies and consequences , 2009, Scientometrics.

[17]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[18]  P. Bonacich.  Factoring and weighting approaches to status scores and clique identification , 1972 .

[19]  Michel Zitt,et al.  Behind citing-side normalization of citations: some properties of the journal impact factor , 2011, Scientometrics.

[20]  Mike Thelwall,et al.  Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows , 2014, J. Assoc. Inf. Sci. Technol..

[21]  E. Garfield.  Citation indexes for science. A new dimension in documentation through association of ideas. 1955. , 1955, International journal of epidemiology.

[22]  P. Gross,et al.  COLLEGE LIBRARIES AND CHEMICAL EDUCATION. , 1927, Science.

[23]  Nees Jan van Eck,et al.  How to normalize cooccurrence data? An analysis of some well-known similarity measures , 2009, J. Assoc. Inf. Sci. Technol..

[24]  Gabriel Pinski,et al.  Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics , 1976, Inf. Process. Manag..

[25]  Henry G. Small,et al.  Clustering thescience citation index® using co-citations , 1985, Scientometrics.

[26]  Anita Coleman,et al.  Assessing the value of a journal beyond the impact factor , 2007, J. Assoc. Inf. Sci. Technol..

[27]  David Adam,et al.  Citation analysis: The counting house , 2002, Nature.

[28]  Henk F. Moed,et al.  Journal impact measures in bibliometric research , 2004, Scientometrics.

[29]  Vicente P. Guerrero-Bote,et al.  A new approach to the metric of journals' scientific prestige: The SJR indicator , 2010, J. Informetrics.

[30]  Péter Jacsó,et al.  Multidimensional Journal Evaluation: Analyzing Scientific Periodicals beyond the Impact Factor , 2013, Online Inf. Rev..

[31]  Jan M. Rabaey,et al.  Comparison of Methods , 2004 .

[32]  E. Garfield,et al.  Citation indexes for science. , 1956, Science.

[33]  R. Cagan.  The San Francisco Declaration on Research Assessment , 2013, Disease Models & Mechanisms.

[34]  Ed C. M. Noyons,et al.  Automatic term identification for bibliometric mapping , 2008, Scientometrics.

[35]  Stefanie Haustein,et al.  Multidimensional Journal Evaluation - Analyzing Scientific Periodicals beyond the Impact Factor , 2012 .

[36]  S. Bradford.  "Sources of information on specific subjects" by S.C. Bradford , 1985 .

[37]  Eugene Garfield,et al.  New factors in the evaluation of scientific literature through citation indexing , 1963 .

[38]  Thed N. van Leeuwen,et al.  A new classification system to describe the ageing of scientific journals and their impact factors , 1998, J. Documentation.

[39]  Henk F. Moed,et al.  Improving the accuracy of Institute for Scientific Information's journal impact factors , 1995 .

[40]  Henk F. Moed,et al.  Measuring contextual citation impact of scientific journals , 2009, J. Informetrics.

[41]  Henry G. Small,et al.  Clustering thescience citation index® using co-citations - I. A comparison of methods , 1985, Scientometrics.

[42]  Yael Edan,et al.  An Individual Feed Allocation Decision Support System for the Dairy Farm , 2001 .

[43]  Rodrigo Costas,et al.  What is the impact of the publications read by the different Mendeley users? Could they help to identify alternative types of impact? PLoS ALM Workshop, San Francisco , 2013 .

[44]  Christian Schlögl,et al.  Dimensionen der Zeitschriftenszientometrie am Beispiel von Buch und Bibliothek , 2006 .

[45]  Wolfgang Glänzel,et al.  Journal citation measures: a concise review , 1988, J. Inf. Sci..

[46]  Carol Tenopir,et al.  Viewing and reading behaviour in a virtual environment: The full-text download and what can be read into it , 2008, Aslib Proc..

[47]  Mike Thelwall,et al.  Validating online reference managers for scholarly impact measurement , 2011, Scientometrics.

[48]  M. Way,et al.  The San Francisco Declaration on Research Assessment , 2013, Journal of Cell Science.

[49]  Ian Rowlands,et al.  Aslib Proceedings is 60 years old , 2008 .

[50]  Vincent Larivière,et al.  Who reads research articles? An altmetrics analysis of Mendeley user categories , 2015, J. Assoc. Inf. Sci. Technol..

[51]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[52]  Thed N. van Leeuwen,et al.  Development and application of journal impact measures in the Dutch science system , 2002, Scientometrics.

[53]  William Gunn,et al.  Social Signals Reflect Academic Impact: What it Means When a Scholar Adds a Paper to Mendeley , 2013 .

[54]  Vincent Larivière,et al.  Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature , 2013, J. Assoc. Inf. Sci. Technol..

[55]  Brian Hayes,et al.  Counting House , 2019, Chaucer.

[56]  Thomas E. Nisonger,et al.  JASIS and Library and Information Science Journal Rankings: A Review and Analysis of the Last Half-Century , 1999, J. Am. Soc. Inf. Sci..

[57]  van Eck Nees Jan,et al.  How to normalize cooccurrence data An analysis of some well-known similarity measures , 2009 .

[58]  Helmut Schmidt,et al.  Probabilistic part-of-speech tagging using decision trees , 1994 .

[59]  Michel Zitt,et al.  Citing-side normalization of journal impact: A robust variant of the Audience Factor , 2010, J. Informetrics.

[60]  Ronald Rousseau,et al.  Journal Evaluation: Technical and Practical Issues, , 2002, Libr. Trends.