A SURVEY ON PERFORMANCE EVALUATION MEASURES FOR INFORMATION RETRIEVAL SYSTEM

information to the users. To make the search effective, a tool called search engine has been introduced. These engines crawl the web for the given users query and display the results to the user based on the relevance score (ranking). Different search engine employs different ranking algorithm. Many ranking algorithm is being introduced frequently by several researchers. Several metrics are available to assess the quality of the ranked web pages. This paper presents a survey on different evaluation measures that are available for information retrieval systems and search engines. Several illustrations are provided for all these metrics.

[1]  Amanda Spink,et al.  Searching the Web: the public and their queries , 2001 .

[2]  James Allan,et al.  A comparison of statistical significance tests for information retrieval evaluation , 2007, CIKM '07.

[3]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[4]  Olivier Chapelle,et al.  Expected reciprocal rank for graded relevance , 2009, CIKM.

[5]  Ramesh Nallapati,et al.  Evaluating the Quality of Query Refinement Suggestions in Information Retrieval , 2006 .

[6]  Harris Wu,et al.  Evaluating Web-based Question Answering Systems , 2002, LREC.

[7]  Yuanzhi Li,et al.  A Theoretical Analysis of NDCG Ranking Measures , 2013 .

[8]  Falk Scholer,et al.  User performance versus precision measures for simple search tasks , 2006, SIGIR.

[9]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[10]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[11]  Robert M. Losee.  When information retrieval measures agree about the relative quality of document rankings , 2000, J. Am. Soc. Inf. Sci..

[12]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[13]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[14]  Joachim M. Buhmann,et al.  The Balanced Accuracy and Its Posterior Distribution , 2010, 2010 20th International Conference on Pattern Recognition.

[15]  Jacek Gwizdka,et al.  Towards Information Retrieval Measures for Evaluation of Web Search Engines , 1999 .

[16]  Djoerd Hiemstra,et al.  Exploiting Query Structure and Document Structure to Improve Document Retrieval Effectiveness , 2006 .

[17]  Tie-Yan Liu,et al.  Learning to Rank for Information Retrieval , 2011 .

[18]  W. Bruce Croft,et al.  Search Engines - Information Retrieval in Practice , 2009 .

[19]  Tie-Yan Liu,et al.  A Theoretical Analysis of NDCG Type Ranking Measures , 2013, COLT.