Learning question classifiers: the role of semantic information

To respond correctly to a free form factual question given a large collection of text data, one needs to understand the question to a level that allows determining some of the constraints the question imposes on a possible answer. These constraints may include a semantic classification of the sought after answer and may even suggest using different strategies when looking for and verifying a candidate answer. This work presents a machine learning approach to question classification. Guided by a layered semantic hierarchy of answer types, we develop a hierarchical classifier that classifies questions into fine-grained classes. This work also performs a systematic study of the use of semantic information sources in natural language classification tasks. It is shown that, in the context of question classification, augmenting the input of the classifier with appropriate semantic category information results in significant improvements to classification accuracy. We show accurate results on a large collection of free-form questions used in TREC 10 and 11.

[1]  W. Lehnert.  A Conceptual Theory of Question Answering , 1986, IJCAI.

[2]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[3]  N. Littlestone.  Mistake bounds and logarithmic linear-threshold learning algorithms , 1990 .

[4]  Steven Abney,et al.  Parsing By Chunks , 1991 .

[5]  Robert C. Berwick,et al.  Principle-Based Parsing: Computation and Psycholinguistics , 1991 .

[6]  Dan Roth,et al.  Learning to Resolve Natural Language Ambiguities: A Unified Approach , 1998, AAAI/IAAI.

[7]  Michael Collins,et al.  AT&T at TREC-8 , 1999, TREC.

[8]  Lynette Hirschman,et al.  Deep Read: A Reading Comprehension System , 1999, ACL.

[9]  Lillian Lee,et al.  Measures of Distributional Similarity , 1999, ACL.

[10]  Sanda M. Harabagiu,et al.  FALCON: Boosting Knowledge for Answer Engines , 2000, TREC.

[11]  Dan Roth,et al.  The Use of Classifiers in Sequential Inference , 2001, NIPS.

[12]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[13]  Ulf Hermjakob,et al.  Parsing and Question Classification for Question Answering , 2001, ACL 2001.

[14]  Eduard H. Hovy,et al.  Toward Semantics-Based Answer Pinpointing , 2001, HLT.

[15]  Dan Roth,et al.  A Sequential Model for Multi-Class Classification , 2001, EMNLP.

[16]  Ellen M. Voorhees,et al.  Overview of the TREC 2002 Question Answering Track , 2003, TREC.

[17]  Performance Issues and Error Analysis in an Open-Domain Question Answering System , 2002, ACL.

[18]  Dan Roth,et al.  Learning Question Classifiers , 2002, COLING.

[19]  Patrick Pantel,et al.  Discovering word senses from text , 2002, KDD.

[20]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[21]  Harris Wu,et al.  Probabilistic question answering on the web , 2002, WWW '02.

[22]  Dan Roth,et al.  Question-Answering via Enhanced Understanding of Questions , 2002, TREC.

[23]  Wei Li,et al.  QuASM: a system for question answering using semi-structured data , 2002, JCDL '02.

[24]  Dell Zhang,et al.  Question classification using support vector machines , 2003, SIGIR.

[25]  Wayne H. Ward,et al.  Question Classification with Support Vector Machines and Error Correcting Codes , 2003, HLT-NAACL.

[26]  Dan Roth,et al.  The Role of Semantic Information in Learning Question Classifiers , 2004 .