Making Stone Soup: Evaluating a Recall-Oriented Multi-stream Question Answering System for Dutch

We describe the participation of the University of Amsterdam in the Question Answering track at CLEF 2004. We took part in the monolingual Dutch task and, for the first time, also in the bilingual English to Dutch task. This year's system is a further elaboration and refinement of the multi-stream architecture we introduced last year, extended with improved candidate answer re-ranking and filtering, and with additional answer finding strategies. We report the evaluation results for the whole system and its various components. The results indicate the recall-oriented approach to QA is an effective one.

[1]  Thorsten Brants,et al.  TnT – A Statistical Part-of-Speech Tagger , 2000, ANLP.

[2]  Valentin Jijkoun,et al.  Information Extraction for Question Answering: Improving Recall Through Syntactic Patterns , 2004, COLING.

[3]  Maarten de Rijke,et al.  Shallow Morphological Analysis in Monolingual Information Retrieval for Dutch, German, and Italian , 2001, CLEF.

[4]  Gilad Mishne,et al.  How frogs built the Berlin Wall , 2004 .

[5]  Jimmy J. Lin,et al.  AskMSR: Question Answering Using the Worldwide Web , 2002 .

[6]  Gertjan van Noord,et al.  Alpino: Wide-coverage Computational Analysis of Dutch , 2000, CLIN.

[7]  M. de Rijke,et al.  Tequesta: The University of Amsterdam's Textual Question Answering System , 2001, TREC.

[8]  Nelleke Oostdijk,et al.  The spoken Dutch Corpus. Outline and first evaluation , 2000 .

[9]  Sanda M. Harabagiu,et al.  LCC Tools for Question Answering , 2002, TREC.

[10]  M. de Rijke,et al.  Using the Corpus Gesproken Nederlands to Build a named Entity Recognizer , 2003 .

[11]  Jimmy J. Lin,et al.  Web question answering: is more always better? , 2002, SIGIR '02.

[12]  M. de Rijke,et al.  Type Checking in Open-Domain Question Answering , 2004, ECAI.

[13]  Nelleke Oostdijk,et al.  The Spoken Dutch Corpus. Overview and First Evaluation , 2000, LREC.

[14]  Carol Peters,et al.  Comparative Evaluation of Multilingual Information Access Systems , 2003, Lecture Notes in Computer Science.

[15]  Valentin Jijkoun,et al.  Answer Selection in a Multi-stream Open Domain Question Answering System , 2004, ECIR.

[16]  Carol Peters,et al.  Evaluation of Cross-Language Information Retrieval Systems , 2002, Lecture Notes in Computer Science.

[17]  Gilad Mishne,et al.  The University of Amsterdam at the TREC 2003 Question Answering Track , 2003, TREC.

[18]  Ellen M. Voorhees,et al.  The Tenth Text REtrieval Conference, TREC 2001 | NIST , 2002 .

[19]  Gilad Mishne,et al.  Preprocessing documents to answer Dutch questions , 2003 .