Problems with the SNIP indicator

As is well known, citation practices differ across academic fields, especially between the science, social science and arts and humanities domains. Thus, when using citations as a measure of research impact, whether for journals, individuals or departments/institutions, it is necessary to normalise the raw data to the general citation potential (Garfield, 1972, 1979) of the research area. Traditionally, this has been done by normalising to the number of citations generally in the field (“cited side” normalisation (Schubert & Braun, 1986)). The most widely used method, developed by CWTS (Moed, 2010a; van Raan, Moed, & van Leeuwen, 2007; van Raan, 2003, 2005), known as the Leiden methodology, compared citation rates per paper to the mean of such rates across a defined research field. Typically, the list of journals in the field was provided by Web of Science (Mingers and Lipitakis, 2013). More recently, an approach was developed (Moed, 2010b; Zitt & Small, 2008) that normalised against the mean number of references per citing paper in the field (“citing side” normalisation). It was also innovatory in not using a pre-defined definition of the relevant field. Rather, the journal’s1 subject field “is defined as the collection of papers citing that journal” (Moed, 2010b, p. 267). More specifically, the subject field is the set of papers that, in a particular year, cite at least one paper in the journal in the preceding ten years (p. 275).2 This approach is called “source-normalised impact per paper (SNIP). This approach was developed in conjunction with Elsevier and was implemented in their Scopus database for all journals contained in it. It is essentially a normalised impact factor for a journal and is becoming widely used.However, problems with this indicator were highlighted by Leydesdorff (2013), Leydesdorff and Opthof (2010a,b) (discussed below) and CWTS, which itself recognised other difficulties. This led to a revised version of SNIP being presented in 2013 (Waltman, van Eck, van Leeuwen, & Visser, 2013). The purpose of this letter is to point out problems with SNIP both in its revised form and the older form.