论文信息 - Overview of the CLEF 2004 Multilingual Question Answering Track

Overview of the CLEF 2004 Multilingual Question Answering Track

Following the pilot Question Answering Track at CLEF 2003, a new evaluation exercise for multilingual QA systems took place in 2004. This paper reports on the novelties introduced in the new campaign and on participants' results. Almost all the cross-language combinations between nine source languages and seven target languages were exploited to set up more than fifty different tasks, both monolingual and bilingual. New types of questions (How- questions and definition questions) were given as input to the participating systems, while just one exact answer per question was allowed as output. The evaluation exercise has highlighted some difficulties in assessing definition questions and can be improved in the future, but the overall analysis of submissions shows encouraging results.

[1] Karen Sparck Jones. Is question answering a rational task , 2003 .

[2] Proceedings of The Eleventh Text REtrieval Conference, TREC 2002, Gaithersburg, Maryland, USA, November 19-22, 2002 , 2002, TREC.

[3] Maarten de Rijke,et al. Creating the DISEQuA Corpus: a Test Set for Multilingual Question Answering , 2003, CLEF.

[4] Maarten de Rijke,et al. The Multiple Language Question Answering Track at CLEF 2003 , 2003, CLEF.

[5] Ellen M. Voorhees,et al. The Tenth Text REtrieval Conference, TREC 2001 | NIST , 2002 .

[6] Ellen M. Voorhees,et al. Overview of the TREC 2002 Question Answering Track , 2003, TREC.

[7] M. Felisa Verdejo,et al. Question Answering Pilot Task at CLEF 2004 , 2004, CLEF.

[8] Carol Peters,et al. Comparative Evaluation of Multilingual Information Access Systems , 2003, Lecture Notes in Computer Science.

[9] Carol Peters,et al. Multilingual Information Access for Text, Speech and Images, 5th Workshop of the Cross-Language Evaluation Forum, CLEF 2004, Bath, UK, September 15-17, 2004, Revised Selected Papers , 2005, CLEF.

[10] Diana Santos,et al. The Key to the First CLEF with Portuguese: Topics, Questions and Answers in CHAVE , 2004, CLEF.

[11] Valentin Jijkoun,et al. Overview of the CLEF 2006 Multilingual Question Answering Track , 2006, CLEF.

[12] M. de Rijke,et al. Creating the DISEQuA corpus: a multilingual test set for the monolingual question Answering tasks at CLEF 2003 , 2003 .