Evaluating Answer Validation in Multi-stream Question Answering

We follow the opinion that Question Answering (QA) performance can be improved by combining different systems. Thus, we planned an evaluation oriented to promote the specialization and further collaboration between QA systems. This multistream QA requires to develop the modules able to select the proper stream according to the question and the candidate answers provided. We describe here the evaluation framework we have developed with special focus on the evaluation measures and the study of their behavior in a comparative evaluation.

[1]  John D. Burger,et al.  MITRE's Qanda at TREC 15 , 2006, TREC.

[2]  M. Felisa Verdejo,et al.  Testing the Reasoning for Question Answering Validation , 2008, J. Log. Comput..

[3]  Jimmy J. Lin,et al.  Data-Intensive Question Answering , 2001, TREC.

[4]  Eduard H. Hovy,et al.  Question Answering in Webclopedia , 2000, TREC.

[5]  Jennifer Chu-Carroll,et al.  In Question Answering, Two Heads Are Better Than One , 2003, NAACL.

[6]  M. Felisa Verdejo,et al.  Overview of the Answer Validation Exercise 2006 , 2006, CLEF.

[7]  Dragomir R. Radev,et al.  Question-answering by predictive annotation , 2000, SIGIR '00.

[8]  Sanda M. Harabagiu,et al.  Methods for Using Textual Entailment in Open-Domain Question Answering , 2006, ACL.

[9]  Sanda M. Harabagiu,et al.  The Structure and Performance of an Open-Domain Question Answering System , 2000, ACL.

[10]  Valentin Jijkoun,et al.  Answer Selection in a Multi-stream Open Domain Question Answering System , 2004, ECIR.

[11]  M. Felisa Verdejo,et al.  Overview of the Answer Validation Exercise 2007 , 2006, CLEF.

[12]  Valentin Jijkoun,et al.  Overview of the CLEF 2007 Multilingual Question Answering Track , 2007, CLEF.

[13]  M. Felisa Verdejo,et al.  Overview of the Answer Validation Exercise 2007 , 2007, CLEF.

[14]  Valentin Jijkoun,et al.  Overview of the CLEF 2006 Multilingual Question Answering Track , 2006, CLEF.

[15]  daniel. sonntag.  Distributed NLP and Machine Learning for Question Answering Grid , 2004 .