Altmetric scores: An early indicator of research impact

In this article we test whether metrics of online attention describing research can provide information on research quality and societal impact that is not found in citation data alone. Our approach is to set up a traditional model in which the true quality or impact of a university department is determined by a panel of experts, but a citation metric is regarded as a reasonable proxy. However, the model assumes that the information contained in the scores provided by an expert panel exceeds that contained in a citation metric (HEFCE, 2015). Finally, we extend this model by including altmetric data to see if it adds information about a department's performance that cannot be gleaned from citations alone. We find the presence of altmetric data for the cited underpinning research to be highly correlated with peer review scores for societal impact. Conversely, no such connection was seen with the assessment of research quality. Our findings therefore suggest altmetric data could be useful as an aid to assessing impact.

[1]  Revolutionizing peer review? , 2005, Nature Neuroscience.

[2]  Tetsuo Tomiyama,et al.  Research Trends , 2018, IntCAD.

[3]  Bradley M. Hemminger,et al.  Altmetrics in the wild: Using social media to explore scholarly impact , 2012, ArXiv.

[4]  Sheila Wilson,et al.  Research Excellence Framework , 2013 .

[5]  Jeffrey M. Woodbridge.  Econometric Analysis of Cross Section and Panel Data , 2002 .

[6]  Lutz Bornmann,et al.  Scientific peer review , 2011, Annu. Rev. Inf. Sci. Technol..

[7]  Lutz Bornmann,et al.  What is societal impact of research and how can it be assessed? a literature survey , 2013, J. Assoc. Inf. Sci. Technol..

[8]  Rodrigo Costas,et al.  Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective , 2014, J. Assoc. Inf. Sci. Technol..

[9]  Mike Thelwall,et al.  Are wikipedia citations important evidence of the impact of scholarly articles and books? , 2017, J. Assoc. Inf. Sci. Technol..

[10]  Sharon Dyas-Correia,et al.  Altmetrics: Rethinking the Way We Measure , 2013 .

[11]  M. Castells,et al.  The Future of Journalism: Networked Journalism , 2012 .

[12]  Maria Liakata,et al.  Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements , 2017, PloS one.

[13]  Jonathan Adams,et al.  The nature, scale and beneficiaries of research impact , 2015 .

[14]  Lutz Bornmann,et al.  How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data , 2016, Scientometrics.

[15]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[16]  Martin Fenner,et al.  Altmetrics and Other Novel Measures for Scientific Impact , 2014 .

[17]  Mike Thelwall,et al.  Evaluating altmetrics , 2013, Scientometrics.

[18]  J. Tobin.  Estimation of Relationships for Limited Dependent Variables , 1958 .

[19]  Mike Thelwall,et al.  The metric tide: report of the independent review of the role of metrics in research assessment and management , 2015 .

[20]  Cameron Barnes,et al.  The Use of Altmetrics as a Tool for Measuring Research Impact , 2015 .

[21]  R. Scoble,et al.  Assessment, evaluations, and definitions of research impact: A review , 2014 .