How to improve the prediction based on citation impact percentiles for years shortly after the publication date?

The findings of Bornmann, Leydesdorff, and Wang (2013b) revealed that the consideration of journal impact improves the prediction of long-term citation impact. This paper further explores the possibility of improving citation impact measurements on the base of a short citation window by the consideration of journal impact and other variables, such as the number of authors, the number of cited references, and the number of pages. The dataset contains 475,391 journal papers published in 1980 and indexed in Web of Science (WoS, Thomson Reuters), and all annual citation counts (from 1980 to 2010) for these papers. As an indicator of citation impact, we used percentiles of citations calculated using the approach of Hazen (1914). Our results show that citation impact measurement can really be improved: If factors generally influencing citation impact are considered in the statistical analysis, the explained variance in the long-term citation impact can be much increased. However, this increase is only visible when using the years shortly after publication but not when using later years.

[1]  J. Hardin,et al.  Generalized Linear Models and Extensions , 2001 .

[2]  Jian Wang,et al.  Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100) , 2013, J. Informetrics.

[3]  Mike Thelwall,et al.  Determinants of research citation impact in nanoscience and nanotechnology , 2013, J. Assoc. Inf. Sci. Technol..

[4]  Lutz Bornmann,et al.  How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects , 2013, J. Informetrics.

[5]  Frauke Kreuter,et al.  Data Analysis Using Stata , 2005 .

[6]  J. Guzmán.  Regression Models for Categorical Dependent Variables Using Stata , 2013 .

[7]  Lutz Bornmann,et al.  Multilevel-statistical reformulation of citation-based university rankings: The Leiden ranking 2011/2012 , 2013, J. Assoc. Inf. Sci. Technol..

[8]  Loet Leydesdorff,et al.  The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000 , 2012, J. Informetrics.

[9]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[10]  Jian Wang,et al.  Citation time window choice for research impact evaluation , 2013, Scientometrics.

[11]  M. Taborsky,et al.  Biased Citation Practice and Taxonomic Parochialism , 2009 .

[12]  Lutz Bornmann,et al.  The problem of citation impact assessments for recent publication years in institutional evaluations , 2013, J. Informetrics.

[13]  Lutz Bornmann,et al.  Scientific peer review , 2011, Annu. Rev. Inf. Sci. Technol..

[14]  Elizabeth S. Vieira,et al.  Citations to scientific articles: Its distribution and dependence on the article features , 2010, J. Informetrics.

[15]  Loet Leydesdorff,et al.  The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011 , 2011, J. Informetrics.

[16]  J. S. Long,et al.  Regression models for categorical dependent variables using Stata, 2nd Edition , 2005 .

[17]  Vicente P. Guerrero-Bote,et al.  A further step forward in measuring journals' scientific prestige: The SJR2 indicator , 2012, J. Informetrics.

[18]  Rex B. Kline,et al.  Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research , 2004 .

[19]  Thed N. van Leeuwen,et al.  The Leiden ranking 2011/2012: Data collection, indicators, and interpretation , 2012, J. Assoc. Inf. Sci. Technol..

[20]  Allen Hazen,et al.  Storage to be Provided Impounding Reservoirs for Municipal Water Supply , 1913 .

[21]  Loet Leydesdorff,et al.  The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits , 2012, J. Informetrics.

[22]  Harvey Goldstein,et al.  League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance , 1996 .

[23]  Stevan Harnad,et al.  Validating research performance metrics against peer rankings , 2008 .

[24]  Rob J Hyndman,et al.  Sample Quantiles in Statistical Packages , 1996 .