On the overlap, the precision and estimated recall of search engines. A case study of the query “Erdos”

In this paper we investigate the retrieval capabilities of six Internet search engines on a simple query. As a case study the query “Erdos” was chosen. Paul Erdos was a world famous Hungarian mathematician, who passed away in September 1996. Existing work on search engine evaluation considers only the first ten or twenty results returned by the search engine, therefore approximation of the recalls of the engines has not been considered so far. In this work we retrieved all 6681 documents that the search engines pointed at and thoroughly examined them. Thus we could calculate the precision of the whole retrieval process, study the overlap between the results of the engines and give an estimate on the recall of the searches. The precision of the engines is high, recall in very low and the overlap is minimal.

[1]  F. W. Lancaster,et al.  Information retrieval: on-line , 1973 .

[2]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[3]  K. Krippendorff.  Krippendorff, Klaus, Content Analysis: An Introduction to its Methodology . Beverly Hills, CA: Sage, 1980. , 1980 .

[4]  Stan A. Hannah,et al.  Communicating globally: the advent of Unicode , 1995 .

[5]  Diana L. Lomarcan,et al.  Networks: the basics , 1995 .

[6]  Hao-hua Chu,et al.  Search En-gines for the World Wide Web: A Compara-tive Study and Evaluation Methodology , 1996 .

[7]  G TomaiuoloNicholas,et al.  An analysis of Internet search engines , 1996 .

[8]  Allison Woodruff,et al.  An Investigation of Documents from the World Wide Web , 1996, Comput. Networks.

[9]  Martin P. Courtois.  Cool tools for web searching An update , 1996 .

[10]  Gary Marchionini,et al.  A Comparative Study of Web Search Service Performance , 1996 .

[11]  Nicholas G. Tomaiuolo,et al.  An analysis of Internet search engines: assessment of over 200 search queries , 1996 .

[12]  Ray R. Larson,et al.  Bibliometrics of the World Wide Web: An Exploratory Analysis of the Intellectual Structure of Cyberspace , 1996 .

[13]  Susan Feldman,et al.  "Just the Answers, Please": Choosing a Web Search Service. , 1997 .

[14]  Brewster Kahle,et al.  Preserving the Internet , 1997 .

[15]  Aggi Raeder,et al.  Financial and Investment Sources on the Web. , 1997 .

[16]  Bruno Oudet,et al.  MULTILINGUALISM ON THE INTERNET , 1997 .

[17]  Peter Ingwersen,et al.  Informetric analyses on the world wide web: methodological approaches to 'webometrics' , 1997, J. Documentation.

[18]  Christine A. DeZelar-Tiedman,et al.  Known-item searching on the World Wide Web , 1997 .

[19]  Carol Ebbinghouse,et al.  Virtuous Funding for the Virtual Library: The Annual SCOUG Retreat, 1997. , 1997 .

[20]  Xiaoying Dong,et al.  SEARCH ENGINES ON THE WORLD WIDE WEB AND INFORMATION RETRIEVAL FROM THE INTERNET: A REVIEW AND EVALUATION , 1997 .

[21]  R. Rousseau.  Sitations: an exploratory study , 1997 .

[22]  B. Danet,et al.  MULTILINGUALISM ON THE INTERNET , 2003 .

[23]  Judit Bar-Ilan,et al.  The “mad cow disease”, Usenet Newsgroups and bibliometric laws , 1997, Scientometrics.