The Measurement of the Effect on Citation Inequality of Differences in Citation Practices across Scientific Fields

This paper has two aims: (i) to introduce a novel method for measuring which part of overall citation inequality can be attributed to differences in citation practices across scientific fields, and (ii) to implement an empirical strategy for making meaningful comparisons between the number of citations received by articles in 22 broad fields. The number of citations received by any article is seen as a function of the article’s scientific influence, and the field to which it belongs. A key assumption is that articles in the same quantile of any field citation distribution have the same degree of citation impact in their respective field. Using a dataset of 4.4 million articles published in 1998–2003 with a five-year citation window, we estimate that differences in citation practices between the 22 fields account for 14% of overall citation inequality. Our empirical strategy is based on the strong similarities found in the behavior of citation distributions. We obtain three main results. Firstly, we estimate a set of average-based indicators, called exchange rates, to express the citations received by any article in a large interval in terms of the citations received in a reference situation. Secondly, using our exchange rates as normalization factors of the raw citation data reduces the effect of differences in citation practices to, approximately, 2% of overall citation inequality in the normalized citation distributions. Thirdly, we provide an empirical explanation of why the usual normalization procedure based on the fields’ mean citation rates is found to be equally successful.

[1]  Loet Leydesdorff,et al.  Integrated Impact Indicators (I3) compared with Impact Factors (IFs): An alternative research design with policy implications , 2011, J. Assoc. Inf. Sci. Technol..

[2]  Thed N. van Leeuwen,et al.  Some modifications to the SNIP journal impact indicator , 2012, J. Informetrics.

[3]  W. Glänzel,et al.  Scientometric indicators. A 32 country comparison of publication productivity and citation impact , 1985 .

[4]  Javier Ruiz-Castillo,et al.  Field normalization at different aggregation levels , 2012 .

[5]  Gabriel Pinski,et al.  Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics , 1976, Inf. Process. Manag..

[6]  Henry G. Small,et al.  Clustering thescience citation index® using co-citations , 1985, Scientometrics.

[7]  Jean-François Molinari,et al.  A new methodology for ranking scientific institutions , 2008, Scientometrics.

[8]  Peter Vinkler,et al.  Evaluation of some methods for the relative assessment of scientific publications , 1986, Scientometrics.

[9]  Juan A. Crespoa,et al.  THE COMPARISON OF CITATION IMPACT ACROSS SCIENTIFIC FIELDS ” , 2012 .

[10]  Claudio Castellano,et al.  Universality of citation distributions: Toward an objective measure of scientific impact , 2008, Proceedings of the National Academy of Sciences.

[11]  Thed N. van Leeuwen,et al.  New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications , 1995, Scientometrics.

[12]  Pedro Albarrán,et al.  References made and citations received by scientific articles , 2011, J. Assoc. Inf. Sci. Technol..

[13]  H. Moed,et al.  The use of bibliometric data for the measurement of university research performance , 1985 .

[14]  A. Shorrocks,et al.  Inequality Decomposition by Population Subgroups , 1984 .

[15]  Pedro Albarrán,et al.  The skewness of science in 219 sub-fields and a number of aggregates , 2010, Scientometrics.

[16]  Tibor Braun,et al.  Cross-field normalization of scientometric indicators , 1996, Scientometrics.

[17]  Loet Leydesdorff,et al.  Normalization at the field level: fractional counting of citations , 2010, J. Informetrics.

[18]  Claudio Castellano,et al.  Testing the fairness of citation indicators for comparison across scientific domains: The case of fractional citation counts , 2011, J. Informetrics.

[19]  E. Balestri,et al.  Global Change and Response of Coastal Dune Plants to the Combined Effects of Increased Sand Accretion (Burial) and Nutrient Availability , 2012, PloS one.

[20]  Tibor Braun,et al.  AGAINST ABSOLUTE METHODS: RELATIVE SCIENTOMETRIC INDICATORS AND RELATIONAL CHARTS AS EVALUATION TOOLS , 1988 .

[21]  Michael J. Moravcsik,et al.  Variation of the nature of citation measures with journals and scientific specialties , 1978, J. Am. Soc. Inf. Sci..

[22]  Wolfgang Glänzel,et al.  Subject field characteristic citation scores and scales for assessing research performance , 1987, Scientometrics.

[23]  Henry G. Small,et al.  Clustering the science citation index using co-citations. II. Mapping science , 1985, Scientometrics.

[24]  A. Shorrocks,et al.  The Class of Additively Decomposable Inequality Measures , 1980 .

[25]  Ludo Waltman,et al.  A new methodology for constructing a publication-level classification system of science , 2012, J. Assoc. Inf. Sci. Technol..

[26]  François Bourguignon,et al.  Decomposable Income Inequality Measures , 1979 .

[27]  Loet Leydesdorff,et al.  Turning the tables in citation analysis one more time: Principles for comparing sets of documents by using an “Integrated Impact Indicator” (I3) , 2011 .

[28]  Michel Zitt,et al.  Modifying the journal impact factor by fractional citation weighting: The audience factor , 2008, J. Assoc. Inf. Sci. Technol..

[29]  Michel Zitt,et al.  Citing-side normalization of journal impact: A robust variant of the Audience Factor , 2010, J. Informetrics.

[30]  Peter Vinkler.  Relations of relative scientometric indicators , 2004, Scientometrics.

[31]  Claudio Castellano,et al.  A Reverse Engineering Approach to the Suppression of Citation Biases Reveals Universal Properties of Citation Distributions , 2012, PloS one.

[32]  Henk F. Moed,et al.  Measuring contextual citation impact of scientific journals , 2009, J. Informetrics.

[33]  A. F. J. van Raan,et al.  Handbook of quantitative studies of science and technology , 1988 .

[34]  Wolfgang Glänzel,et al.  The application of characteristic scores and scales to the evaluation and ranking of scientific journals , 2011, J. Inf. Sci..

[35]  Koenraad Debackere,et al.  A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking , 2011, Scientometrics.

[36]  Javier Ruiz-Castillo,et al.  Multiplicative and Fractional Strategies When Journals are Assigned to Several Sub-Fields , 2011, J. Assoc. Inf. Sci. Technol..

[37]  Tibor Braun,et al.  Relative indicators and relational charts for comparative assessment of publication output and citation impact , 1986, Scientometrics.

[38]  Lutz Bornmann,et al.  Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization , 2011, J. Informetrics.