Comparing journal and paper level classifications of science

Abstract The classification of science into disciplines is at the heart of bibliometric analyses. While most classifications systems are implemented at the journal level, their accuracy has been questioned, and paper-level classifications have been considered by many to be more precise. However, few studies investigated the difference between journal and the paper classification systems. This study addresses this gap by comparing the journal- and paper-level classifications for the same set of papers and journals. This isolates the effects of classification precision (i.e., journal- or paper-level) to reveal the extent of paper misclassification. Results show almost half of papers could be misclassified in journal classification systems. Given their importance in the construction and analysis of bibliometric indicators, more attention should be given to the robustness and accuracy of these disciplinary classifications schemes.

[1]  Kevin W. Boyack,et al.  Including cited non-source items in a large-scale map of science: What difference does it make? , 2014, J. Informetrics.

[2]  Julia Melkers,et al.  Bibliometrics as a Tool for Analysis of R&D Impacts , 1993 .

[3]  Yuen-Hsien Tseng,et al.  Journal clustering of library and information science for subfield delineation using the bibliometric analysis toolkit: CATAR , 2013, Scientometrics.

[4]  Kevin W. Boyack,et al.  Which Type of Citation Analysis Generates the Most Accurate Taxonomy of Scientific and Technical Knowledge? , 2015, J. Assoc. Inf. Sci. Technol..

[5]  Kevin W. Boyack,et al.  Clustering More than Two Million Biomedical Publications: Comparing the Accuracies of Nine Text-Based Similarity Approaches , 2011, PloS one.

[6]  Loet Leydesdorff,et al.  The operationalization of “fields” as WoS subject categories (WCs) in evaluative bibliometrics: The cases of “library and information science” and “science & technology studies” , 2014, J. Assoc. Inf. Sci. Technol..

[7]  Ludo Waltman,et al.  A new methodology for constructing a publication-level classification system of science , 2012, J. Assoc. Inf. Sci. Technol..

[8]  Kevin W. Boyack,et al.  Creation of a highly detailed, dynamic, global model and map of science , 2014, J. Assoc. Inf. Sci. Technol..

[9]  B. C. Griffith,et al.  The Structure of Scientific Literatures II: Toward a Macro- and Microstructure for Science , 1974 .

[10]  Ismael Rafols,et al.  A global map of science based on the ISI subject categories , 2009 .

[11]  Wolfgang Glänzel,et al.  Diversity of references as an indicator of the interdisciplinarity of journals: Taking similarity between subject fields into account , 2016, J. Assoc. Inf. Sci. Technol..

[12]  Wolfgang Glänzel,et al.  Journal cross-citation analysis for validation and improvement of journal-based subject classification in bibliometric research , 2010, Scientometrics.

[13]  Ismael Rafols,et al.  Content-based and algorithmic classifications of journals: Perspectives on the dynamics of scientific communication and indexer effects , 2009 .

[14]  Kevin W. Boyack,et al.  Research Portfolio Analysis and Topic Prominence , 2017, J. Informetrics.

[15]  Alexander I. Pudovkin,et al.  Algorithmic procedure for finding semantically related journals , 2002, J. Assoc. Inf. Sci. Technol..

[16]  Kevin W. Boyack,et al.  A principled methodology for comparing relatedness measures for clustering publications , 2019, ISSI.

[17]  Wolfgang Glänzel,et al.  A new classification scheme of science fields and subfields designed for scientometric evaluation purposes , 2004, Scientometrics.

[18]  Kevin W. Boyack,et al.  Co-citation analysis, bibliographic coupling, and direct citation: Which citation approach represents the research front most accurately? , 2010 .

[19]  B. C. Griffith,et al.  The Structure of Scientific Literatures I: Identifying and Graphing Specialties , 1974 .

[20]  Fei Shu,et al.  Mapping science using Library of Congress Subject Headings , 2017, J. Informetrics.

[21]  Fernando Deschamps,et al.  Assessing the maturity of a research area: bibliometric review and proposed framework , 2016, Scientometrics.

[22]  Bing Wang,et al.  Chinese science citation database: Its construction and application , 1999, Scientometrics.

[23]  Éric Archambault,et al.  The decline in the concentration of citations, 1900-2007 , 2008, J. Assoc. Inf. Sci. Technol..

[24]  Bart De Moor,et al.  Hybrid clustering for validation and improvement of subject-classification schemes , 2009, Inf. Process. Manag..

[25]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[26]  V. Larivière,et al.  Design and Update of a Classification System: The UCSD Map of Science , 2012, PloS one.

[27]  Shih-Chieh Fang,et al.  Tracking R&D behavior: bibliometric analysis of drug patents in the Orange Book , 2011, Scientometrics.

[28]  Pei-Shan Chi,et al.  Differing disciplinary citation concentration patterns of book and journal literature? , 2016, J. Informetrics.

[29]  Henry Small,et al.  A SYSTEM FOR AUTOMATIC CLASSIFICATION OF SCIENTIFIC LITERATURE , 2013 .

[30]  N. D. Bellis.  Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics , 2009 .