Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data

The aim of this study is to measure differences between three relevant altmetric providers: Altmetric.com, PlumX and Crossref Event Data (CED). More than 67,000 research papers, initially extracted from PlumX, were searched in Altmetric.com and CED to compare their counts. Differences between services were analyzed regarding the number of documents with an altmetric event and the counting differences in each metric. Results show that Altmetric.com is the provider with the best coverage of blog posts, news and tweets; while PlumX better collects Mendeley readers; and CED is the site that extracts more Wikipedia citations. The study concludes that there are important counting differences due to technical errors and extracting criteria. The article recommends the use of specific services for the analysis of particular metrics. While, it should be mandatory to employ the combination of several providers, if we want to perform an overall analysis.

[1]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[2]  Rodrigo Costas,et al.  General discussion of data quality challenges in social media metrics: Extensive comparison of four major altmetric data aggregators , 2018, PloS one.

[3]  J. Michael Lindsay,et al.  PlumX from Plum Analytics: Not Just Altmetrics , 2016 .

[4]  Rodrigo Costas,et al.  Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective , 2014, J. Assoc. Inf. Sci. Technol..

[5]  Lutz Bornmann,et al.  Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime , 2014, J. Informetrics.

[6]  Tobias Siebenlist,et al.  Cross-metric compatability and inconsistencies of altmetrics , 2018, Scientometrics.

[7]  Yin Leng Theng,et al.  Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media , 2016, Scientometrics.

[8]  Peter Kraker,et al.  Research Data Explored: Citations versus Altmetrics , 2015, ISSI.

[9]  Vincent Larivière,et al.  Scholarly use of social media and altmetrics: A review of the literature , 2016, J. Assoc. Inf. Sci. Technol..

[10]  José Luis Ortega,et al.  To be or not to be on Twitter, and its relationship with the tweeting and citation of research papers , 2016, Scientometrics.

[11]  Sarah L. Shreeves,et al.  Connecting the pieces: Using ORCIDs to improve research impact and repositories , 2015, F1000Research.

[12]  Nickoal Eichmann-Kalwara.  Research data explored: an extended analysis of citations and altmetrics , 2017 .

[13]  Uc San Francisco,et al.  The Altmetric Score: A New Measure for Article-Level Dissemination and Impact , 2015 .

[14]  Kim Holmberg,et al.  Altmetrics for Information Professionals: Past, Present and Future , 2015 .

[15]  Nicolás Robinson-García,et al.  New data, new possibilities: Exploring the insides of Altmetric.com , 2014, ArXiv.

[16]  Zohreh Zahedi,et al.  What do we know about Altmetric.com sources? A study of the top 200 blogs and news sites mentioning scholarly output. Altmetrics workshop, 9 October 2015, Amsterdam Science Park, Amsterdam , 2015 .

[17]  Jane Burns.  Altmetrics: a Practical Guide for Librarians, Researchers, and Academics , 2018, DBS Business Review.

[18]  Euan A. Adie,et al.  Altmetric: enriching scholarly content with article‐level discussion and metrics , 2013, Learn. Publ..

[19]  Christian Pieter Hoffmann,et al.  Altmetrics for large, multidisciplinary research groups : Comparison of current tools , 2014 .

[20]  Daniel Torres-Salinas,et al.  PlumX As a Potential Tool to Assess the Macroscopic Multidimensional Impact of Books , 2017, Front. Res. Metr. Anal..

[21]  Scott Chamberlain,et al.  Consuming Article-Level Metrics: Observations and Lessons , 2013 .

[22]  Wolfgang Glänzel,et al.  The ecstasy and the agony of the altmetric score , 2016, Scientometrics.

[23]  Peter Kraker,et al.  Research Data Explored II: the Anatomy and Reception of figshare , 2015, ArXiv.

[24]  Rodrigo Costas,et al.  Consistency among altmetrics data provider/aggregators : What are the challenges? Altmetrics workshop, 9 October 2015, Amsterdam Science Park, Amsterdam , 2015 .