Patent citation analysis with Google

Citations from patents to scientific publications provide useful evidence about the commercial impact of academic research, but automatically searchable databases are needed to exploit this connection for large‐scale patent citation evaluations. Google covers multiple different international patent office databases but does not index patent citations or allow automatic searches. In response, this article introduces a semiautomatic indirect method via Bing to extract and filter patent citations from Google to academic papers with an overall precision of 98%. The method was evaluated with 322,192 science and engineering Scopus articles from every second year for the period 1996–2012. Although manual Google Patent searches give more results, especially for articles with many patent citations, the difference is not large enough to be a major problem. Within Biomedical Engineering, Biotechnology, and Pharmacology & Pharmaceutics, 7% to 10% of Scopus articles had at least one patent citation but other fields had far fewer, so patent citation analysis is only relevant for a minority of publications. Low but positive correlations between Google Patent citations and Scopus citations across all fields suggest that traditional citation counts cannot substitute for patent citations when evaluating research.

[1]  Debora Shaw,et al.  Web citation data for impact assessment: A comparison of four science disciplines , 2005, J. Assoc. Inf. Sci. Technol..

[2]  Mike Thelwall,et al.  Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis , 2007, J. Assoc. Inf. Sci. Technol..

[3]  Koenraad Debackere,et al.  Do science-technology interactions pay off when developing technology? , 2004, Scientometrics.

[4]  Liwen Vaughan,et al.  Web citation data for impact assessment: A comparison of four science disciplines: Book Reviews , 2005 .

[5]  Bart Van Looy,et al.  Sources of inspiration? Making sense of scientific references in patents , 2014, Scientometrics.

[6]  Wen-Chi Hung Measuring the use of public research in firm R&D in the Hsinchu Science Park , 2012, Scientometrics.

[7]  S. D. De Groote,et al.  Coverage of Google Scholar, Scopus, and Web of Science: a case study of the h-index in nursing. , 2012, Nursing outlook.

[8]  Mike Thelwall,et al.  How is science cited on the Web? A classification of google unique Web citations , 2007, J. Assoc. Inf. Sci. Technol..

[9]  Debora Shaw,et al.  Bibliographic and Web citations: What is the difference? , 2003, J. Assoc. Inf. Sci. Technol..

[10]  Henk F. Moed,et al.  An exploration of the science base of recent technology , 1990 .

[11]  Sadao Nagaoka,et al.  Assessing the R&D Management of a Firm in Terms of Speed and Science Linkage: Evidence from the U.S. Patents , 2007 .

[12]  Sungjoo Lee,et al.  An approach to discovering new technology opportunities: Keyword-based patent map approach , 2009 .

[13]  Péter Jacsó,et al.  Google Scholar duped and deduped – the aura of “robometrics” , 2011 .

[14]  Yi-Ching Liaw,et al.  Can the technological impact of academic journals be evaluated? The practice of non-patent reference (NPR) analysis , 2014, Scientometrics.

[15]  Kimberly S. Hamilton,et al.  The increasing linkage between U.S. technology and public science , 1997 .

[16]  Mike Thelwall,et al.  Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses , 2008, J. Assoc. Inf. Sci. Technol..

[17]  Ying He,et al.  Patent-bibliometric analysis on the Chinese science — technology linkages , 2007, Scientometrics.

[18]  Jean Pierre Courtial,et al.  The use of patent titles for identifying the topics of invention and forecasting trends , 1993, Scientometrics.

[19]  Bart Van Looy,et al.  Delineating the scientific footprint in technology: Identifying scientific publications within non-patent references , 2011, Scientometrics.

[20]  Ulrich Schmoch,et al.  Tracing the knowledge transfer from science to technology as reflected in patent indicators , 2005, Scientometrics.

[21]  Mike Thelwall,et al.  Online presentations as a source of scientific impact? An analysis of PowerPoint files citing academic journals , 2008, J. Assoc. Inf. Sci. Technol..

[22]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[23]  M. Gittelman,et al.  Applicant and Examiner Citations in US Patents: An Overview and Analysis , 2008 .

[24]  Mike Thelwall,et al.  Guideline references and academic citations as evidence of the clinical value of health research , 2016, J. Assoc. Inf. Sci. Technol..

[25]  Yuen-Hsien Tseng,et al.  Text mining techniques for patent analysis , 2007, Inf. Process. Manag..

[26]  Lijun Zhu,et al.  Mining Technical Topic Networks from Chinese Patents , 2014, IPaMin@KONVENS.

[27]  M. Gittelman,et al.  Patent Citations as a Measure of Knowledge Flows: The Influence of Examiner Citations , 2006, The Review of Economics and Statistics.

[28]  Gobinda G. Chowdhury,et al.  Automatic extraction of citations from the text of English-language patents - an example of template mining , 1996, J. Inf. Sci..

[29]  P. Collins,et al.  Citations in patents to the basic research literature , 1988 .

[30]  Min Deng,et al.  The evidence of systematic noise in non-patent references: A study of New Zealand companies’ patents , 2007, Scientometrics.

[31]  Mike Thelwall,et al.  An automatic method for assessing the teaching impact of books from online academic syllabi , 2016, J. Assoc. Inf. Sci. Technol..

[32]  Mark A. Lemley,et al.  Examiner Characteristics and Patent Office Outcomes , 2009, Review of Economics and Statistics.

[33]  Francis Narin,et al.  Is technology becoming science? , 1985, Scientometrics.

[34]  Mike Thelwall,et al.  An automatic method for extracting citations from Google Books , 2015, J. Assoc. Inf. Sci. Technol..

[35]  Jacques Michel,et al.  Patent citation analysis.A closer look at the basic input data from patent search reports , 2001, Scientometrics.

[36]  Nabil Amara,et al.  Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science , 2012, Scientometrics.

[37]  Francis Narin,et al.  Status report: Linkage between technology and science , 1992 .

[38]  Wipo World Intellectual Property Indicators, 2017 edition , 2017 .

[39]  Nicolás Robinson-García,et al.  The Google scholar experiment: How to index false papers and manipulate bibliometric indicators , 2013, J. Assoc. Inf. Sci. Technol..

[40]  Koenraad Debackere,et al.  Can applied science be ‘good science’? Exploring the relationship between patent citations and citation impact in nanoscience , 2010, Scientometrics.

[41]  Joost C. F. de Winter,et al.  The expansion of Google Scholar versus Web of Science: a longitudinal study , 2013, Scientometrics.

[42]  Mike Thelwall,et al.  Google book search: Citation analysis for social science and the humanities , 2009, J. Assoc. Inf. Sci. Technol..

[43]  Loet Leydesdorff,et al.  The university-industry knowledge relationship: Analyzing patents and the science base of technologies , 2004, J. Assoc. Inf. Sci. Technol..

[44]  Mike Thelwall,et al.  Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus , 2011, J. Assoc. Inf. Sci. Technol..

[45]  Yuanyuan Ma,et al.  Computer aided system of screening and sorting in data processing for non-patent literature , 2010, 2010 International Conference on Computer Application and System Modeling (ICCASM 2010).

[46]  Blaise Cronin,et al.  Bibliometrics and beyond: some thoughts on web-based citation analysis , 2001, J. Inf. Sci..

[47]  Martin Meyer,et al.  Academic patents as an indicator of useful research? A new approach to measure academic inventiveness , 2003 .

[48]  Annapoornima M. Subramanian,et al.  An empirical examination of the science-technology relationship in the biotechnology industry , 2010 .

[49]  Thed N. van Leeuwen,et al.  Technological Relevance of Science: An Assessment of Citation Linkages between Patents and Research Papers , 2000, Scientometrics.

[50]  M. Meyer Does science push technology? Patents citing scientific literature , 2000 .

[51]  Manabu Okumura,et al.  Automatic extraction of citation information in Japanese patent applications , 2008, International Journal on Digital Libraries.

[52]  Koenraad Debackere,et al.  Linking science to technology: Using bibliographic references in patents to build linkage schemes , 2004, Scientometrics.

[53]  Mu-Hsuan Huang,et al.  Technological impact factor: An indicator to measure the impact of academic publications on practical innovation , 2014, J. Informetrics.

[54]  Jöran Beel,et al.  On the robustness of google scholar against spam , 2010, HT '10.

[55]  A. Kulkarni,et al.  Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. , 2009, JAMA.

[56]  Mike Thelwall,et al.  The influence of time and discipline on the magnitude of correlations between citation counts and quality scores , 2015, J. Informetrics.

[57]  Wesley M. Cohen Fifty Years of Empirical Studies of Innovative Activity and Performance , 2010 .

[58]  R. Tijssen Global and domestic utilization of industrial relevant science: patent citation analysis of science-technology interactions and knowledge flows , 2001 .

[59]  Guo Zhang,et al.  Patent citation analysis: Calculating science linkage based on citing motivation , 2014, J. Assoc. Inf. Sci. Technol..

[60]  Patrice Lopez Automatic Extraction and Resolution of Bibliographical References in Patent Documents , 2010, IRFC.

[61]  M. Keary The Web of Knowledge: A Festschrift in Honor of Eugene Garfield , 2001 .

[62]  R. Liuzzi Knowledge Bases , 2001 .

[63]  Byungun Yoon,et al.  A text-mining-based patent network: Analytical tool for high-technology trend , 2004 .

[64]  Stefano Brusoni,et al.  The knowledge bases of the world’s largest pharmaceutical groups: what do patent citations to non-patent literature reveal? , 2005 .

[65]  Madian Khabsa,et al.  Digital commons , 2020, Internet Policy Rev..

[66]  Judit Bar-Ilan,et al.  Which h-index? — A comparison of WoS, Scopus and Google Scholar , 2008, Scientometrics.

[67]  Koenraad Debackere,et al.  Traces of Prior Art: An analysis of non-patent references found in patent documents , 2006, Scientometrics.

[68]  Anne-Wil Harzing,et al.  Google Scholar as a new source for citation analysis , 2008 .

[69]  Szu-chia S. Lo,et al.  Scientific linkage of science research and technology development: a case of genetic engineering research , 2009, Scientometrics.

[70]  Martin Meyer,et al.  What is Special about Patent Citations? Differences between Scientific and Patent Citations , 2000, Scientometrics.

[71]  Masashi Shirabe Identifying SCI covered publications within non-patent references in U.S. utility patents , 2014, Scientometrics.