Query evaluation for suitable search engine selection

Web Search Engines (SEs) have gained much attention in the last decade. More especially, traditional SEs such as Google, aids web searchers extensively in federated search, as a preliminary tool to locate the right vertical (specialized SEs like in the case of travel-sites) in order to launch their queries. The retrieval and ranking approaches differ across SEs and besides that, the basic underlying knowledge about each SE is not sufficient enough to make the best SE selection for different query categories. Therefore, how do searchers identify the most suitable SE that optimizes their query needs? In this paper, we evaluate three famous SEs from a front-end perspective using 7 sub-categories query-sets from BaseQuery and RelationalQuery. We apply the Normalized Discounted Cumulative Gain (NDCG), Mean Reciprocal Rank (MRR) and our proposed Robustness metric to assess the SEs. Using a 2 tailed t-test, we show that there is a significant difference among the SEs in5 query categories, in terms of relevant retrieval and ranking.

[1]  Hinrich Schütze,et al.  Introduction to Information Retrieval: Scoring, term weighting, and the vector space model , 2008 .

[2]  Walid Magdy,et al.  Examining the Robustness of Evaluation Metrics for Patent Retrieval with Incomplete Relevance Judgements , 2010, CLEF.

[3]  Dirk Lewandowski,et al.  The retrieval effectiveness of search engines on navigational queries , 2011, Aslib Proc..

[4]  Rakesh Chandra Balabantaray.  Evaluation of web search engine based on ranking of results and its features , 2017 .

[5]  Sanjay K. Dwivedi,et al.  Search engines evaluation using users efforts , 2011, 2011 2nd International Conference on Computer and Communication Technology (ICCCT-2011).

[6]  Martin 0. Andago,et al.  Evaluation of a Semantic Search Engine against a Keyword Search Engine Using First 20 Precision 1 , 2011 .

[7]  Jaana Kekäläinen,et al.  Binary and graded relevance in IR evaluations--Comparison of the effects on ranking of IR systems , 2005, Inf. Process. Manag..

[8]  Baagyere Edward Yellakuor,et al.  Towards ranking cultural terms from originating source , 2016, 2016 Third International Conference on Digital Information Processing, Data Mining, and Wireless Communications (DIPDMWC).

[9]  Ricardo A. Baeza-Yates,et al.  Query Recommendation Using Query Logs in Search Engines , 2004, EDBT Workshops.

[10]  Peter Bailey,et al.  Measuring Search Engine Quality , 2001, Information Retrieval.

[11]  Jiawei Han,et al.  Data Mining for Web Intelligence , 2002, Computer.

[12]  Amanda Spink,et al.  A study of results overlap and uniqueness among major Web search engines , 2006, Inf. Process. Manag..

[13]  Judit Bar-Ilan,et al.  Comparing rankings of search results on the Web , 2005, Inf. Process. Manag..

[14]  Amanda Spink,et al.  An Analysis of Web Documents Retrieved and Viewed , 2003, International Conference on Internet Computing.

[15]  Srikanta Patnaik,et al.  Intelligent Computing, Networking, and Informatics, Proceedings of the International Conference on Advanced Computing, Networking, and Informatics, ICACNI 2013, Raipur, Chhattisgarh, India, 12-14 June 2013 , 2014, ICACNI.

[16]  Bela Mutschler,et al.  On the Precision of Search Engines: Results from a Controlled Experiment , 2012, BIS.

[17]  Daqing Zhang,et al.  Where is the Largest Market: Ranking Areas by Popularity from Location Based Social Networks , 2013, 2013 IEEE 10th International Conference on Ubiquitous Intelligence and Computing and 2013 IEEE 10th International Conference on Autonomic and Trusted Computing.

[18]  Hinrich Schütze,et al.  Introduction to Information Retrieval: Language models for information retrieval , 2008 .

[19]  Alfred Glossbrenner,et al.  Search engines for the World Wide Web , 1997 .

[20]  Antal van den Bosch,et al.  Estimating search engine index size variability: a 9-year longitudinal study , 2016, Scientometrics.

[21]  Shaojie Qiao,et al.  SimRank: A Page Rank approach based on similarity measure , 2010, 2010 IEEE International Conference on Intelligent Systems and Knowledge Engineering.