论文信息 - On the Evaluation of Snippet Selection for WebCLEF

On the Evaluation of Snippet Selection for WebCLEF

WebCLEF is about supporting a user who is an expert in writing a survey article on a specific topic with a clear goal and audience by generating a ranked list with relevant snippets. This paper focuses on the evaluation methodology of WebCLEF. We show that the evaluation method and test set used for WebCLEF 2007 cannot be used to evaluate new systems and give recommendations how to improve the evaluation.

[1] James A. Thom,et al. HiXEval: Highlighting XML Retrieval Evaluation , 2005, INEX.

[2] Ellen M. Voorhees,et al. The TREC-8 Question Answering Track Evaluation , 2000, TREC.

[3] Carol Peters,et al. Advances in Multilingual and Multimodal Information Retrieval, 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, September 19-21, 2007, Revised Selected Papers , 2008, CLEF.

[4] Gabriella Kazai,et al. Advances in XML Information Retrieval and Evaluation, 4th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2005, Dagstuhl Castle, Germany, November 28-30, 2005, Revised Selected Papers , 2006, INEX.

[5] Chin-Yew Lin,et al. ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[6] Valentin Jijkoun,et al. Overview of WebCLEF 2007 , 2008, CLEF.

[7] Daniel E. Rose,et al. Understanding user goals in web search , 2004, WWW '04.

[8] Valentin Jijkoun,et al. Using Centrality to Rank Web Snippets , 2008, CLEF.