Substance without citation: evaluating the online impact of grey literature

Individuals and organisations producing information or knowledge for others sometimes need to be able to provide evidence of the value of their work in the same way that scientists may use journal impact factors and citations to indicate the value of their papers. There are many cases, however, when organisations are charged with producing reports but have no real way of measuring their impact, including when they are distributed free, do not attract academic citations and their sales cannot be tracked. Here, the web impact report (WIRe) is proposed as a novel solution for this problem. A WIRe consists of a range of web-derived statistics about the frequency and geographic location of online mentions of an organisation’s reports. WIRe data is typically derived from commercial search engines. This article defines the component parts of a WIRe and describes how to collect and analyse the necessary data. The process is illustrated with a comparison of the web impact of the reports of a large UK organisation. Although a formal evaluation was not conducted, the results suggest that WIRes can indicate different levels of web impact between reports and can reveal the type of online impact that the reports have.

[1]  William A. Gerber.  The Internet in Britain 2009 , 2009 .

[2]  Peter Ingwersen,et al.  The calculation of web impact factors , 1998, J. Documentation.

[3]  Mike Thelwall,et al.  Web issue analysis: An integrated water resource management case study: Research Articles , 2006 .

[4]  Frank R. Baumgartner,et al.  Punctuated equilibrium theory and environmental policy , 2006 .

[5]  Blaise Cronin,et al.  Invoked on the Web , 1998, J. Am. Soc. Inf. Sci..

[6]  Kimberly A. Neuendorf,et al.  The Content Analysis Guidebook , 2001 .

[7]  魏屹东,et al.  Scientometrics , 2018, Encyclopedia of Big Data.

[8]  Debora Shaw,et al.  Bibliographic and Web citations: What is the difference? , 2003, J. Assoc. Inf. Sci. Technol..

[9]  Lutz Bornmann,et al.  What is societal impact of research and how can it be assessed? a literature survey , 2013, J. Assoc. Inf. Sci. Technol..

[10]  M. Thelwall,et al.  Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis , 2007 .

[11]  Elena Maceviciute.  Review of: Thelwall, Michael. Introduction to webometrics: quantitative web research for the social sciences. San Rafael, CA: Morgan & Claypool, 2009 , 2010, Inf. Res..

[12]  KoushaKayvan,et al.  Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis , 2007 .

[13]  Mike Thelwall.  Extracting accurate and complete results from search engines: Case study windows live , 2008 .

[14]  June A. Flora,et al.  Social Marketing and Public Health Intervention , 1988, Health education quarterly.

[15]  Mike Thelwall,et al.  Web issue analysis: An integrated water resource management case study , 2006, J. Assoc. Inf. Sci. Technol..

[16]  Mike Thelwall,et al.  Motivations for academic web site interlinking: evidence for the Web as a novel source of information on informal scholarly communication , 2003, J. Inf. Sci..

[17]  Debora Shaw,et al.  Banking (on) different forms of symbolic capital , 2002, J. Assoc. Inf. Sci. Technol..

[18]  Roger Bennett,et al.  Sources and use of marketing information by marketing managers , 2007, J. Documentation.

[19]  L. Butler,et al.  Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts , 2003 .

[20]  Mike Thelwall,et al.  Introduction to Webometrics: Quantitative Web Research for the Social Sciences , 2009, Introduction to Webometrics.

[21]  Ahmet Uyar,et al.  Investigation of the accuracy of search engine hit counts , 2009, J. Inf. Sci..

[22]  Charles Oppenheim,et al.  The Influence of Peer Review on the Research Assessment Exercise , 2004, J. Inf. Sci..

[23]  Salvatore Mele,et al.  Citing and reading behaviours in high-energy physics , 2009, Scientometrics.

[24]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[25]  Salvatore Mele,et al.  Citing and Reading Behaviors of High-Energy Physics or How a Community Stopped Worrying about Journals and Learned to Love Repositories , 2009 .

[26]  Mike Thelwall,et al.  A generic lexical URL segmentation framework for counting links, colinks or URLs , 2008 .

[27]  Ronald N. Kostoff,et al.  The metrics of science and technology , 2001, Scientometrics.

[28]  H. Moed.  Citation Analysis in Research Evaluation (Information Science & Knowledge Management) , 2005 .

[29]  Keith G. Jeffery,et al.  An architecture for grey literature in a R&D context , 2000 .