A word clustering approach for language model-based sentence retrieval in question answering systems

In this paper we propose a term clustering approach to improve the performance of sentence retrieval in Question Answering (QA) systems. As the search in question answering is conducted over smaller segments of data than in a document retrieval task, the problems of data sparsity and exact matching become more critical. In this paper we propose Language Modeling (LM) techniques to overcome such problems and improve the sentence retrieval performance. Our proposed methods include building class-based models by term clustering, and then employing higher order n-grams with the new class-based model. We report our experiments on the TREC 2007 questions from QA track. The results show that the methods investigated here enhanced the mean average precision of sentence retrieval from 23.62% to 29.91%.

[1]  Masaki Aono,et al.  A Method for Query Expansion Using a Hierarchy of Clusters , 2005, AIRS.

[2]  Dan Shen,et al.  Exploring rich evidence for maximum entropy-based question answering , 2008 .

[3]  W. Bruce Croft,et al.  A general language model for information retrieval (poster abstract) , 1999, SIGIR '99.

[4]  Andreas Merkel,et al.  Comparing Improved Language Models for Sentence Retrieval inQuestion Answering , 2007 .

[5]  Thorsten Brants,et al.  Distributed Word Clustering for Large Scale Class-Based Language Modeling in Machine Translation , 2008, ACL.

[6]  Victoria J. Hodge,et al.  Hierarchical word clustering - automatic thesaurus generation , 2002, Neurocomputing.

[7]  John D. Lafferty,et al.  A study of smoothing methods for language models applied to Ad Hoc information retrieval , 2001, SIGIR '01.

[8]  Jimmy J. Lin,et al.  Overview of the TREC 2007 Question Answering Track , 2008, TREC.

[9]  Dietrich Klakow,et al.  Exploring Syntactic Relation Patterns for Question Answering , 2005, IJCNLP.

[10]  Wolfgang Reichl,et al.  A class-based language model for large-vocabulary speech recognition extracted from part-of-speech statistics , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[11]  W. Bruce Croft,et al.  TREC and Tipster Experiments with Inquery , 1995, Inf. Process. Manag..

[12]  Jianfeng Gao,et al.  Dependence language model for information retrieval , 2004, SIGIR '04.

[13]  Robert L. Mercer,et al.  Class-Based n-gram Models of Natural Language , 1992, CL.

[14]  Wang Huizhen,et al.  Automatic word clustering for text categorization using global information , 2004 .

[15]  Tianshun Yao,et al.  Automatic Word Clustering for Text Categorization Using Global Information , 2004, AIRS.

[16]  W. Bruce Croft,et al.  A general language model for information retrieval , 1999, CIKM '99.

[17]  Hang Li,et al.  Word Clustering and Disambiguation Based on Co-occurrence Data , 1998, COLING.