Question answering passage retrieval using dependency relations

State-of-the-art question answering (QA) systems employ term-density ranking to retrieve answer passages. Such methods often retrieve incorrect passages as relationships among question terms are not considered. Previous studies attempted to address this problem by matching dependency relations between questions and answers. They used strict matching, which fails when semantically equivalent relationships are phrased differently. We propose fuzzy relation matching based on statistical models. We present two methods for learning relation mapping scores from past QA pairs: one based on mutual information and the other on expectation maximization. Experimental results show that our method significantly outperforms state-of-the-art density-based passage retrieval methods by up to 78% in mean reciprocal rank. Relation matching also brings about a 50% improvement in a system enhanced by query expansion.

[1]  Robert L. Mercer,et al.  The Mathematics of Statistical Machine Translation: Parameter Estimation , 1993, CL.

[2]  David A. Hull Using statistical testing in the evaluation of retrieval experiments , 1993, SIGIR.

[3]  Justin Zobel,et al.  Passage retrieval revisited , 1997, SIGIR '97.

[4]  John D. Lafferty,et al.  Information retrieval as statistical translation , 1999, SIGIR '99.

[5]  W. Bruce Croft,et al.  A general language model for information retrieval , 1999, CIKM '99.

[6]  Adwait Ratnaparkhi,et al.  IBM's Statistical Question Answering System , 2000, TREC.

[7]  Antonio Cisternino,et al.  PiQASso: Pisa Question Answering System , 2001, TREC.

[8]  Patrick Pantel,et al.  Discovery of inference rules for question-answering , 2001, Natural Language Engineering.

[9]  Salim Roukos,et al.  IBM's Statistical Question Answering System-TREC 11 , 2001, TREC.

[10]  Jungyun Seo,et al.  SiteQ: Engineering High Performance QA System Using Lexico-Semantic Pattern Matching and Shallow NLP , 2001, TREC.

[11]  Gideon S. Mann,et al.  Analyses for elucidating current question answering technology , 2001, Natural Language Engineering.

[12]  E. Voorhees Overview of the TREC 2003 Question Answering Track , 2004, TREC.

[13]  Jimmy J. Lin,et al.  What Makes a Good Answer? The Role of Context in Question Answering , 2003, INTERACT.

[14]  Jimmy J. Lin,et al.  Selectively Using Relations to Improve Precision in Question Answering , 2003 .

[15]  Jimmy J. Lin,et al.  Quantitative evaluation of passage retrieval algorithms for question answering , 2003, SIGIR.

[16]  Dekang Lin,et al.  Dependency-Based Evaluation of Minipar , 2003 .

[17]  Daniel Marcu,et al.  A Noisy-Channel Approach to Question Answering , 2003, ACL.

[18]  Sanda M. Harabagiu,et al.  Answer Mining by Combining Extraction Techniques with Abductive Reasoning , 2003, Text Retrieval Conference.

[19]  Jianfeng Gao,et al.  Dependence language model for information retrieval , 2004, SIGIR '04.

[20]  Tat-Seng Chua,et al.  National University of Singapore at the TREC 13 Question Answering Main Task , 2004, TREC.

[21]  Philip Koehn,et al.  Statistical Machine Translation , 2010, EAMT.