Beyond REF 2014: The impact of impact assessment on the future of information research

The importance of demonstrating value for money in terms of academic research beyond the walls of institutions grows stronger as demonstrated by the inclusion of impact assessment in the 2014 REF (Research Excellence Framework) exercise for UK higher education institutions (HEIs). To understand if such focus is influencing the library and information science (LIS) discipline, this paper reports a critical examination of impact case studies submitted to REF 2014 under the Communication, Cultural and Media Studies, Library and Information Management Unit of Assessment. Content analysis was conducted on 25 case studies submitted by 14 institutions, establishing the methodologies, impacts, beneficiaries, published outputs and corroborative evidence reported. The implications of impact assessment on future LIS researcher behaviour, in terms of research conceptualisation and design, were explored through nine qualitative telephone interviews. While individual researchers did not anticipate their behaviour to change due to the introduction of impact assessments, there are anticipated changes across the discipline including a greater focus on engaging with stakeholders and research beneficiaries at early stages of research design and an emphasis on mixed methodologies to maximise the power and consequences of research results.

[1]  John Hudson,et al.  Measuring the benefits of university research: impact and the REF in the UK , 2015 .

[2]  R. Gray Has the Research Excellence Framework killed creativity? , 2015, Journal of psychiatric and mental health nursing.

[3]  Cassidy R. Sugimoto,et al.  Beyond Gatekeepers of Knowledge: Scholarly Communication Practices of Academic Librarians and Archivists at ARL Institutions , 2014, Coll. Res. Libr..

[4]  William H. Walters,et al.  Disciplinary, national, and departmental contributions to the literature of library and information science, 2007–2012 , 2016, J. Assoc. Inf. Sci. Technol..

[5]  Branwen Morgan Research impact: Income for outcome , 2014, Nature.

[6]  Heting Chu Research methods in library and information science: A content analysis , 2015 .

[7]  J. Goddard,et al.  From outcomes to process: evidence for a new approach to research impact assessment , 2014 .

[8]  Vincent Larivière,et al.  The weakening relationship between the impact factor and papers' citations in the digital age , 2012, J. Assoc. Inf. Sci. Technol..

[9]  Liz Allen,et al.  Alternative Perspectives on Impact: The Potential of ALMs and Altmetrics to Inform Funders about Research Impact , 2014, PLoS biology.

[10]  Catriona Manville,et al.  Preparing impact submissions for REF 2014: An evaluation: Findings and observations , 2015 .

[11]  Roger Burrows,et al.  The impact of impact , 2014 .

[12]  Thorsten Gruber Academic sell-out: how an obsession with metrics and rankings is damaging academia , 2014 .

[13]  Catriona Manville,et al.  Assessing Research Impact , 2013 .

[14]  Helen Colley Radical Social Theory for Radical Times: Putting Theory to Work in Educational Research. What(a) to do about impact? A bourdieusian critique , 2012 .

[15]  Paul Solomon,et al.  Looking for Information—A Survey of Research on Information Seeking, Needs, and Behavior , 2003, Information Retrieval.

[16]  Paul Simmonds,et al.  REF Accountability Review: Costs, benefits and burden , 2015 .

[17]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[18]  R. Watermeyer Issues in the articulation of ‘impact’: the responses of UK academics to ‘impact’ as a new measure of research assessment , 2014 .

[19]  Gabrielle Samuel,et al.  Exploring evaluator perceptions of the characterization of impact under the REF2014 , 2015 .

[20]  J. Molas-Gallart Research evaluation and the assessment of public value , 2012 .

[21]  Tony P. Murphy,et al.  Perceptions of the UK’s Research Excellence Framework 2014: a media analysis , 2014 .

[22]  Salil Gunashekar,et al.  Preparing impact submissions for REF 2014 , 2015 .

[23]  Vincent Larivière,et al.  The decade of metrics? Examining the evolution of metrics within and outside LIS , 2012 .

[24]  S. Craig Chaoqun Andrew Cassidy R. Finlay,et al.  Publish or Practice? An Examination of Librarians’ Contributions to Research , 2013 .