Which types of online evidence show the nonacademic benefits of research? Websites cited in UK impact case studies

Abstract While funders increasingly request evidence of the societal benefits of research, all academics in the UK must periodically provide this information to gain part of their block funding within the Research Excellence Framework (REF). The impact case studies produced in the UK are public and can therefore be used to gain insights into the types of sources used to justify societal impact claims. This study focuses on the URLs cited as evidence in the last public REF to help researchers and resource providers to understand what types can be used and the disciplinary differences in their uptake. Based on a new semiautomatic method to classify the URLs cited in impact case studies, the results show that there are a few key online types of source for most broad fields, but these sources differ substantially between subject areas. For example, news websites are more important in some fields than others, and YouTube is sometimes used for multimedia evidence in the arts and humanities. Knowledge of the common sources selected independently by thousands of researchers may help others to identify suitable sources for the complex task of evidencing societal impacts.

[1]  K. Robb,et al.  Evidencing the impact of cancer trials: insights from the 2014 UK Research Excellence Framework , 2020, Trials.

[2]  D. Webber,et al.  Achieving wider impact in business and management: analysing the case studies from REF 2014 , 2019 .

[3]  Clare Wilkinson,et al.  Evidencing impact: a case study of UK academic perspectives on evidencing research impact , 2019 .

[4]  Jenny Wooldridge,et al.  Altmetric scores: An early indicator of research impact , 2018, J. Assoc. Inf. Sci. Technol..

[5]  Lutz Bornmann,et al.  Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (REF) , 2018, J. Informetrics.

[6]  Mike Thelwall,et al.  Springer Handbook of Science and Technology Indicators , 2019, Springer Handbook of Science and Technology Indicators.

[7]  Kayvan Kousha,et al.  Web Citation Indicators for Wider Impact Assessment of Articles , 2019, Springer Handbook of Science and Technology Indicators.

[8]  Katy Jordan,et al.  How was social media cited in 2014 REF Impact Case Studies , 2018 .

[9]  L. Brook Evidencing Impact from Art Research: Analysis of Impact Case Studies from the REF 2014 , 2018 .

[10]  Martin Stepanek,et al.  Guidance for standardising quantitative indicators of impact within REF case studies , 2018 .

[11]  Fiona Ross,et al.  Exploring research impact in the assessment of leadership, governance and management research , 2017 .

[12]  Maria Liakata,et al.  Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements , 2017, PloS one.

[13]  Steven Wooding,et al.  100 Metrics to Assess and Communicate the Value of Biomedical Research: An Ideas Book. , 2017, Rand health quarterly.

[14]  Melanie Smallman,et al.  Beyond Academia – Interrogating Research Impact in the Research Excellence Framework , 2016, PloS one.

[15]  Rita Marcella,et al.  Beyond REF 2014: The impact of impact assessment on the future of information research , 2016, J. Inf. Sci..

[16]  Mike Thelwall,et al.  Alternative metric indicators for funding scheme evaluations , 2016, Aslib J. Inf. Manag..

[17]  T. Greenhalgh,et al.  Research impact in the community-based health sciences: an analysis of 162 case studies from the 2014 UK Research Excellence Framework , 2015, BMC Medicine.

[18]  Mike Thelwall,et al.  The metric tide: report of the independent review of the role of metrics in research assessment and management , 2015 .

[19]  Jonathan Adams,et al.  The nature, scale and beneficiaries of research impact , 2015 .

[20]  Liz Allen,et al.  Alternative Perspectives on Impact: The Potential of ALMs and Altmetrics to Inform Funders about Research Impact , 2014, PLoS biology.

[21]  T. Lorenc,et al.  A systematic review of barriers to and facilitators of the use of evidence by policymakers , 2014, BMC Health Services Research.

[22]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[23]  N. Mays,et al.  Describing the impact of health research: a Research Impact Framework , 2006, BMC Health Services Research.