Link-based approach to study scientific software usage: the case of VOSviewer

Scientific software is a fundamental player in modern science, participating in all stages of scientific knowledge production. Software occasionally supports the development of trivial tasks, while at other instances it determines procedures, methods, protocols, results, or conclusions related with the scientific work. The growing relevance of scientific software as a research product with value of its own has triggered the development of quantitative science studies of scientific software. The main objective of this study is to illustrate a link-based webometric approach to characterize the online mentions to scientific software across different analytical frameworks. To do this, the bibliometric software VOSviewer is used as a case study. Considering VOSviewer’s official website as a baseline, online mentions to this website were counted in three different analytical frameworks: academic literature via Google Scholar (988 mentioning publications), webpages via Majestic (1,330 mentioning websites), and tweets via Twitter (267 mentioning tweets). Google scholar mentions shows how VOSviewer is used as a research resource, whilst mentions in webpages and tweets show the interest on VOSviewer’s website from an informational and a conversational point of view. Results evidence that URL mentions can be used to gather all sorts of online impacts related to non-traditional research objects, like software, thus expanding the analytical scientometric toolset by incorporating a novel digital dimension.

[1]  Adrián A. Díaz-Faes,et al.  Towards a second generation of ‘social media metrics’: Characterizing Twitter communities of attention around science , 2019, PloS one.

[2]  Erjia Yan,et al.  Examining the usage, citation, and diffusion patterns of bibliometric mapping software: A comparative study of three tools , 2018, J. Informetrics.

[3]  Arthur E. Kirkpatrick,et al.  Assessing open source software as a scholarly contribution , 2009, Commun. ACM.

[4]  James Howison,et al.  Software in the scientific literature: Problems with seeing, finding, and using software mentioned in the biology literature , 2016, J. Assoc. Inf. Sci. Technol..

[5]  Enrique Orduña-Malea,et al.  Methods for estimating the size of Google Scholar , 2014, Scientometrics.

[6]  James D. Herbsleb,et al.  Scientific software production: incentives and collaboration , 2011, CSCW.

[7]  Ming Cui,et al.  How important is software to library and information science research? A content analysis of full-text publications , 2019, J. Informetrics.

[8]  Shirley Williams,et al.  What do people study when they study Twitter? Classifying Twitter related academic papers , 2013, J. Documentation.

[9]  Hyoungjoo Park,et al.  Research software citation in the Data Citation Index: Current practices and implications for research software sharing and reuse , 2019, J. Informetrics.

[10]  Michael Gusenbauer,et al.  Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases , 2018, Scientometrics.

[11]  Daniel S. Katz,et al.  The Challenge and Promise of Software Citation for Credit, Identification, Discovery, and Reuse , 2016, ACM J. Data Inf. Qual..

[12]  Heather A. Piwowar,et al.  Altmetrics: Value all research products , 2013, Nature.

[13]  Qianqian Wang,et al.  Assessing the impact of software on science: A bootstrapped learning of software entities in full-text papers , 2015, J. Informetrics.

[14]  Enrique Orduña-Malea,et al.  U.S. academic libraries: understanding their web presence and their relationship with economic indicators , 2013, Scientometrics.

[15]  Erjia Yan,et al.  Disciplinary differences of software use and impact in scientific literature , 2016, Scientometrics.

[16]  Kai Li,et al.  Co-mention network of R packages: Scientific impact and clustering structure , 2018, J. Informetrics.

[17]  Enrique Orduña-Malea,et al.  Google Scholar as a data source for research assessment , 2017, Springer Handbook of Science and Technology Indicators.

[18]  S. Ovadia.  Exploring the Potential of Twitter as a Research Tool , 2009 .

[19]  Kai Li,et al.  How is R cited in research outputs? Structure, impacts, and citation standard , 2017, J. Informetrics.

[20]  Lorraine J. Hwang,et al.  Citations for Software: Providing Identification, Access and Recognition for Research Software , 2017, Int. J. Digit. Curation.

[21]  Emilio Delgado López-Cózar,et al.  Google Scholar as a source for scholarly evaluation: A bibliographic review of database errors , 2017 .

[22]  James Howison,et al.  Softcite dataset: A dataset of software mentions in biomedical and economic research publications , 2021, J. Assoc. Inf. Sci. Technol..

[23]  Mike Thelwall,et al.  Interpreting social science link analysis research: A theoretical framework , 2006, J. Assoc. Inf. Sci. Technol..

[24]  Rodrigo Costas,et al.  Social media metrics for new research evaluation , 2018, Springer Handbook of Science and Technology Indicators.

[25]  Ludo Waltman,et al.  Software survey: VOSviewer, a computer program for bibliometric mapping , 2009, Scientometrics.

[26]  Isidro F. Aguillo,et al.  Size of web domains and interlinking behavior of higher education institutions in Europe , 2014, Scientometrics.

[27]  Kai Li,et al.  Challenges of measuring the impact of software: an examination of the lme4 R package , 2018, J. Informetrics.

[28]  James D. Herbsleb,et al.  Understanding the scientific software ecosystem and its impact: Current and future measures , 2015 .

[29]  Gaël Varoquaux,et al.  Publishing scientific software matters , 2013, J. Comput. Sci..

[30]  Blaise Cronin,et al.  Invoked on the Web , 1998, J. Am. Soc. Inf. Sci..

[31]  Mike Thelwall,et al.  Hyperlink Analyses of the World Wide Web: A Review , 2006, J. Comput. Mediat. Commun..

[32]  Xue Wang,et al.  How important is scientific software in bioinformatics research? A comparative study between international and Chinese research communities , 2018, J. Assoc. Inf. Sci. Technol..

[33]  Axel Bruns,et al.  Twitter data analytics - or: the pleasures and perils of studying Twitter , 2014, Aslib J. Inf. Manag..

[34]  Enrique Orduna-Malea,et al.  Dot-science top level domain: Academic websites or dumpsites? , 2021, Scientometrics.