Improving Question Answering by Combining Multiple Systems Via Answer Validation

Nowadays there exist several kinds of question answering systems. According to recent evaluation results, most of these systems are complementary (i.e., each one is better than the others in answering some specific type of questions). This fact indicates that a pertinent combination of various systems may allow improving the best individual result. This paper focuses on this problem. It proposes using an answer validation method to handle this combination. The main advantage of this approach is that it does not rely on internal system's features nor depend on external answer's redundancies. Experimental results confirm the appropriateness of our proposal. They mainly show that it outperforms individual system's results as well as the precision obtained by a redundancy-based combination strategy.

[1]  Fredric C. Gey,et al.  ENSM-SE at CLEF 2006 : Fuzzy Proximity Method with an Adhoc Influence Function in Evaluation of Multilingual and Multi-modal Information Retrieval 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2006, Alicante, Spain , 2007 .

[2]  Peter Ingwersen,et al.  Developing a Test Collection for the Evaluation of Integrated Search , 2010, ECIR.

[3]  M. Felisa Verdejo,et al.  SPARTE, a Test Suite for Recognising Textual Entailment in Spanish , 2006, CICLing.

[4]  M. Felisa Verdejo,et al.  Overview of the Answer Validation Exercise 2007 , 2007, CLEF.

[5]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[6]  Diego Mollá Aliod,et al.  Extracting Exact Answers using a Meta Question Answering System , 2005, ALTA.

[7]  Jennifer Chu-Carroll,et al.  In Question Answering, Two Heads Are Better Than One , 2003, NAACL.

[8]  M. Felisa Verdejo,et al.  Overview of the Answer Validation Exercise 2006 , 2006, CLEF.

[9]  Bernardo Magnini,et al.  Is It the Right Answer? Exploiting Web Redundancy for Answer Validation , 2002, ACL.

[10]  Manuel Montes-y-Gómez,et al.  Graph-Based Answer Fusion in Multilingual Question Answering , 2007, TSD.

[11]  Thomas G. Dietterich.  Machine-Learning Research Four Current Directions , 1997 .

[12]  Valentin Jijkoun,et al.  Overview of the CLEF 2006 Multilingual Question Answering Track , 2006, CLEF.

[13]  Ido Dagan,et al.  Machine Learning Challenges, Evaluating Predictive Uncertainty, Visual Object Classification and Recognizing Textual Entailment, First PASCAL Machine Learning Challenges Workshop, MLCW 2005, Southampton, UK, April 11-13, 2005, Revised Selected Papers , 2006, MLCW.

[14]  Bogdan Sacaleanu,et al.  Overview of the CLEF 2008 Multilingual Question Answering Track , 2008, CLEF.

[15]  Valentin Jijkoun,et al.  Answer Selection in a Multi-stream Open Domain Question Answering System , 2004, ECIR.

[16]  Diane J. Litman,et al.  Improving Question Answering for Reading Comprehension Tests by Combining Multiple Systems , 2005 .

[17]  Maarten de Rijke,et al.  Overview of the CLEF 2004 Multilingual Question Answering Track , 2004, CLEF.

[18]  Fredric C. Gey,et al.  Accessing Multilingual Information Repositories, 6th Workshop of the Cross-Language Evalution Forum, CLEF 2005, Vienna, Austria, 21-23 September, 2005, Revised Selected Papers , 2006, CLEF.

[19]  Xavier Carreras,et al.  FreeLing: An Open-Source Suite of Language Analyzers , 2004, LREC.