Errors of omission and their implications for computing scientometric measures in evaluating the publishing productivity and impact of countries

Purpose – The purpose of the paper is to explore the extent of the absence of data elements that are critical from the perspective of scientometric evaluation of the scientific productivity and impact of countries in terms of the most common indicators – such as the number of publications, the number of citations and the impact factor (the ratio of citations received to papers published), and the effect these may have on the h‐index of countries – in two of the most widely used citation‐enhanced databases.Design/methodology/approach – The author uses the Scopus database and Thomson‐Reuters' (earlier known as ISI) three citation databases (Science, Social Sciences and Arts & Humanities), both as implemented on the Dialog Information Services (Thomson ISI databases) and on the Web of Knowledge platform, known as Web of Science (WoS). The databases were searched to discover how many records they have for each year, how many of those have cited references for each year, and what percentage of the records have...

[1]  Péter Jacsó,et al.  The pros and cons of computing the h-index using Google Scholar , 2008, Online Inf. Rev..

[2]  Lokman I. Meho,et al.  Using the h-index to rank influential information scientists , 2006, J. Assoc. Inf. Sci. Technol..

[3]  Péter Jacsó,et al.  Content Evaluation of Databases. , 1997 .

[4]  L. Bornmann,et al.  The state of h index research , 2009, EMBO reports.

[5]  Chris Neuhaus,et al.  The Depth and Breadth of Google Scholar: An Empirical Study , 2006 .

[6]  Judit Bar-Ilan,et al.  Some measures for comparing citation databases , 2007, J. Informetrics.

[7]  Linda Butler,et al.  Using a balanced approach to bibliometrics: quantitative performance measures in the Australian Research Quality Framework , 2008 .

[8]  Jerome K. Vanclay,et al.  On the robustness of the h-index , 2007, J. Assoc. Inf. Sci. Technol..

[9]  Lars Iselid,et al.  Web of Science and Scopus: a journal title overlap study , 2008, Online Inf. Rev..

[10]  AntÔnio F. Pereira De Araújo.  Increasing discrepancy between absolute and effective indexes of research output in a Brazilian academic department , 2008 .

[11]  Judit Bar-Ilan,et al.  Which h-index? — A comparison of WoS, Scopus and Google Scholar , 2008, Scientometrics.

[12]  S. Fingerman,et al.  SCOPUS: Profusion and confusion , 2005 .

[13]  Hans-Dieter Daniel,et al.  Data sources for performing citation analysis: an overview , 2008, J. Documentation.

[14]  Péter Jacsó,et al.  Google Scholar revisited , 2008, Online Inf. Rev..

[15]  Loet Leydesdorff,et al.  Caveats for the Use of Citation Indicators in Research and Journal Evaluations , 2008, J. Assoc. Inf. Sci. Technol..

[16]  Ivan Figueira,et al.  Institutional h-index: The performance of a new metric in the evaluation of Brazilian Psychiatric Post-graduation Programs , 2008, Scientometrics.

[17]  Péter Jacsó.  The dimensions of cited reference enhanced database subsets , 2007, Online Inf. Rev..

[18]  Ronald Rousseau,et al.  The influence of missing publications on the Hirsch index , 2007, J. Informetrics.

[19]  Gary E. Gorman.  "They can't read, but they sure can count": Flawed rules of the journal rankings game , 2008, Online Inf. Rev..

[20]  S. Fingerman,et al.  Web of Science and Scopus: Current features and Capabilities. , 2006, Issues in Science and Technology Librarianship.

[21]  Lokman I. Meho,et al.  Citation Analysis: A Comparison of Google Scholar, Scopus, and Web of Science , 2007, Proceedings of the American Society for Information Science and Technology.

[22]  Péter Jacsó,et al.  The pros and cons of computing the h-index using Scopus , 2008, Online Inf. Rev..

[23]  Péter Jacsó,et al.  Comparison and Analysis of the Citedness Scores in Web of Science and Google Scholar , 2005, ICADL.

[24]  Péter Jacsó.  Searching for skeletons in the database cupboard. Part I: errors of omission , 1993 .

[25]  András Schubert,et al.  Hirsch-index for countries based on Essential Science Indicators data , 2007, Scientometrics.

[26]  Zaida Chinchilla-Rodríguez,et al.  Coverage analysis of Scopus: A journal metric approach , 2007, Scientometrics.

[27]  Peter Jasco,et al.  Testing the Calculation of a Realistic h-index in Google Scholar, Scopus, and Web of Science for F. W. Lancaster , 2008 .

[28]  Peter Jacso.  Content Evaluation of Textual CD-ROM and Web Databases. Database Searching Series. , 2001 .

[29]  Bruce White.  Examining the claims of Google Scholar as a serious information source , 2006 .

[30]  Concepción S. Wilson,et al.  Informetric studies using databases: Opportunities and challenges , 2003, Scientometrics.

[31]  Péter Jacsó,et al.  The plausibility of computing the h-index of scholarly productivity and impact using reference-enhanced databases , 2008, Online Inf. Rev..

[32]  Rónán O’Beirne,et al.  Content Evaluation of Textual CD‐ROM and Web Databases , 2002 .

[33]  Andreas Thor,et al.  Convergent validity of bibliometric Google Scholar data in the field of chemistry - Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts , 2009, J. Informetrics.

[34]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[35]  P. Jacsó.  As we may search : Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases , 2005 .

[36]  Charles Oppenheim,et al.  Comparing alternatives to the Web of Science for coverage of the social sciences' literature , 2007, J. Informetrics.

[37]  Péter Jacsó,et al.  Deflated, inflated and phantom citation counts , 2006, Online Inf. Rev..

[38]  Péter Jacsó,et al.  The pros and cons of computing the h-index using Web of Science , 2008, Online Inf. Rev..

[39]  Lutz Bornmann,et al.  Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine , 2008, J. Assoc. Inf. Sci. Technol..