Examining the usage, citation, and diffusion patterns of bibliometric mapping software: A comparative study of three tools

This study investigates the use, citation and diffusion of three bibliometric mapping software tools (CiteSpace, HistCite and VOSviewer) in scientific papers. We first conduct a content analysis of a sample of 481 English core journal papers—i.e., papers from journals deemed central to their respective disciplines—in which at least one of these tools is mentioned. This allows us to understand the predominant mention and citation practices surrounding these tools. We then employ several diffusion indicators to gain insight into the diffusion patterns of the three software tools. Overall, we find that researchers mention and cite the tools in diverse ways, many of which fall short of a traditional formal citation. Our results further indicate a clear upward trend in the use of all three tools, though VOSviewer is more frequently used than CiteSpace or HistCite. We also find that these three software tools have seen the fastest and most widespread adoption in library and information science research, where the tools originated. They have since been gradually adopted in other areas of study, initially at a lower diffusion speed but afterward at a rapidly growing rate.

[1]  R. Cartes-Velásquez,et al.  Bibliometric analysis of articles published in ISI dental journals, 2007–2011 , 2013, Scientometrics.

[2]  Howard D. White,et al.  Pathfinder networks and author cocitation analysis: A remapping of paradigmatic information scientists , 2003, J. Assoc. Inf. Sci. Technol..

[3]  Xiaotong Yu,et al.  How is CiteSpace Used and Cited in the Literature? An Analysis of the Articles Published in English and Chinese Core Journals , 2017, ISSI.

[4]  Min Song,et al.  Entitymetrics: Measuring the Impact of Entities , 2013, PloS one.

[5]  Birger Larsen,et al.  Bibliometric indicators of young authors in astrophysics: Can later stars be predicted? , 2014, Scientometrics.

[6]  Weimao Ke,et al.  Mapping the diffusion of scholarly knowledge among major U.S. research institutions , 2006, Scientometrics.

[7]  Alexander I. Pudovkin,et al.  Why do we need algorithmic historiography? , 2003, J. Assoc. Inf. Sci. Technol..

[8]  Jürgen Harald Jacob,et al.  Early recognition of high quality researchers of the German psychiatry by worldwide accessible bibliometric indicators , 2007, Scientometrics.

[9]  Daniel S. Katz,et al.  Software citation principles , 2016, PeerJ Comput. Sci..

[10]  Deen Freelon ReCal: Intercoder Reliability Calculation as a Web Service , 2010 .

[11]  Mike Thelwall,et al.  Academic software downloads from Google Code: useful usage indicators? , 2016, Inf. Res..

[12]  James Howison,et al.  Software in the scientific literature: Problems with seeing, finding, and using software mentioned in the biology literature , 2016, J. Assoc. Inf. Sci. Technol..

[13]  George A. Barnett,et al.  A Mathematical Model of Academic Citation Age , 1989 .

[14]  Tiffany C. Chao Disciplinary reach: Investigating the impact of dataset reuse in the earth sciences , 2011, ASIST.

[15]  Christopher W. Belter,et al.  Measuring the Value of Research Data: A Citation Analysis of Oceanographic Data Sets , 2014, PloS one.

[16]  Henry G. Small,et al.  Update on science mapping: Creating large document spaces , 1997, Scientometrics.

[17]  Ludo Waltman,et al.  Software survey: VOSviewer, a computer program for bibliometric mapping , 2009, Scientometrics.

[18]  Erjia Yan,et al.  Disciplinary knowledge production and diffusion in science , 2016, J. Assoc. Inf. Sci. Technol..

[19]  C. Urquhart,et al.  A bibliometric approach demonstrates the impact of a social care data set on research and policy. , 2013, Health information and libraries journal.

[20]  Hailey Mooney,et al.  Citing data sources in the social sciences: do authors do it? , 2011, Learn. Publ..

[21]  Evaristo Jiménez-Contreras,et al.  Analyzing data citation practices using the data citation index , 2015, J. Assoc. Inf. Sci. Technol..

[22]  Ian Rowlands,et al.  Journal diffusion factors: a new approach to measuring research influence , 2002, Aslib Proc..

[23]  Vincent Larivière,et al.  The role of handbooks in knowledge creation and diffusion: A case of science and technology studies , 2014, J. Informetrics.

[24]  Bin Zhang,et al.  Tracing database usage: Detecting main paths in database link networks , 2015, J. Informetrics.

[25]  Ludo Waltman,et al.  Visualizing Bibliometric Networks , 2014 .

[26]  Ronald E. Rice,et al.  The Convergence of Information Science and Communication: A Bibliometric Analysis , 1992, J. Am. Soc. Inf. Sci..

[27]  Daniel S. Katz,et al.  The Challenge and Promise of Software Citation for Credit, Identification, Discovery, and Reuse , 2016, ACM J. Data Inf. Qual..

[28]  Erjia Yan,et al.  Disciplinary differences of software use and impact in scientific literature , 2016, Scientometrics.

[29]  Peter Kraker,et al.  Research Data Explored: Citations versus Altmetrics , 2015, ISSI.

[30]  Steven Wooding,et al.  Tracking knowledge diffusion through citations , 2005 .

[31]  Chaomei Chen,et al.  CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature , 2006, J. Assoc. Inf. Sci. Technol..

[32]  Ronald Rousseau,et al.  Knowledge diffusion through publications and citations: A case study using ESI-fields as unit of diffusion , 2010, J. Assoc. Inf. Sci. Technol..

[33]  Francisco Herrera,et al.  Science mapping software tools: Review, analysis, and cooperative study among tools , 2011, J. Assoc. Inf. Sci. Technol..

[34]  Chaomei Chen,et al.  Science Mapping: A Systematic Review of the Literature , 2017, J. Data Inf. Sci..

[35]  Heather A. Piwowar,et al.  Altmetrics: Value all research products , 2013, Nature.

[36]  Ronald Rousseau,et al.  Robert Fairthorne and the empirical power laws , 2005, J. Documentation.

[37]  Chaomei Chen,et al.  Searching for intellectual turning points: Progressive knowledge domain visualization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[38]  C. Borgman Bibliometrics and Scholarly Communication , 1989 .

[39]  Michael W. Morris,et al.  The Lessons We (Don't) Learn: Counterfactual Thinking and Organizational Accountability after a Close Call , 2000 .

[40]  Eugene Garfield,et al.  From the science of science to Scientometrics visualizing the history of science with HistCite software , 2009, J. Informetrics.

[41]  James D. Herbsleb,et al.  Scientific software production: incentives and collaboration , 2011, CSCW.

[42]  Chaomei Chen,et al.  Visualizing knowledge domains , 2005, Annu. Rev. Inf. Sci. Technol..

[43]  Kai Li,et al.  How is R cited in research outputs? Structure, impacts, and citation standard , 2017, J. Informetrics.

[44]  Douglas G. Altman,et al.  Practical statistics for medical research , 1990 .

[45]  Ian Rowlands,et al.  Emerald Article : Diffusion factors , 2022 .

[46]  Arthur E. Kirkpatrick,et al.  Assessing open source software as a scholarly contribution , 2009, Commun. ACM.

[47]  Bart Verspagen,et al.  Knowledge Flows, Patent Citations and the Impact of Science on Technology , 2008 .

[48]  Giovanni Abramo,et al.  National research assessment exercises: a comparison of peer review and bibliometrics rankings , 2011, Scientometrics.

[49]  Thed N. van Leeuwen,et al.  Citation delay in interdisciplinary knowledge exchange , 2001, Scientometrics.

[50]  Guo Zhang,et al.  Content‐based citation analysis: The next generation of citation analysis , 2014, J. Assoc. Inf. Sci. Technol..

[51]  James D. Herbsleb,et al.  Understanding the scientific software ecosystem and its impact: Current and future measures , 2015 .

[52]  Tun Lu,et al.  Meanings and boundaries of scientific software sharing , 2013, CSCW.

[53]  Yuh-Shan Ho,et al.  Comparison of independent research of China’s top universities using bibliometric indicators , 2012, Scientometrics.

[54]  Lily Troia Citations for Software: Providing Identification, Access and Recognition for Research Software , 2017 .

[55]  Thed N. van Leeuwen,et al.  Interdisciplinary dynamics of modern science: analysis of cross-disciplinary citation flows , 2000 .

[56]  Ronald Rousseau,et al.  Towards a representation of diffusion and interaction of scientific ideas: The case of fiber optics communication , 2012, Inf. Process. Manag..

[57]  Ronald Rousseau,et al.  Knowledge diffusion through publications and citations: A case study using ESI-fields as unit of diffusion , 2010 .

[58]  Janice Singer,et al.  How do scientists develop and use scientific software? , 2009, 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering.

[59]  E. Rogers Diffusion of Innovations , 1962 .

[60]  Qianqian Wang,et al.  Assessing the impact of software on science: A bootstrapped learning of software entities in full-text papers , 2015, J. Informetrics.

[61]  Shengnan Wu,et al.  The Network Pattern of Journal Knowledge Transfer in Library and Information Science in China , 2014 .

[62]  Erjia Yan,et al.  Disciplinary knowledge diffusion in business research , 2017, J. Informetrics.