Semantic Chunk Annotation for questions using Maximum Entropy

We present a ME (Maximum Entropy) model for Semantic Chunk Annotation in a Chinese Question and Answer (Q&A) system. The model was derived from a corpus of real world questions, which are collected from some discussion groups on the Internet. The questions are supposed to be answered by other people, so the questions are very complex. The semantic chunks were introduced. Feature for the model was described and MI (mutual information) was adopted for feature selection. The training data consists of 14000 sentences and the test data consists of 4000 sentences. The result: F-score is 90.68%.

[1]  Ulf Hermjakob,et al.  Parsing and Question Classification for Question Answering , 2001, ACL 2001.

[2]  Eric Brill,et al.  Automatic Question Answering: Beyond the Factoid , 2004, NAACL.

[3]  Eric Brill,et al.  Automatic question answering using the web: Beyond the Factoid , 2006, Information Retrieval.

[4]  Sanda M. Harabagiu,et al.  Answering complex questions with random walk models , 2006, SIGIR '06.

[5]  Eric Horvitz,et al.  Using Machine Learning Techniques to Interpret WH-questions , 2001, ACL.

[6]  Diego Molla Aliod,et al.  Question Answering in Restricted Domains: An Overview , 2007, CL.

[7]  Myung-Gil Jang,et al.  Descriptive Question Answering in Encyclopedia , 2005, ACL.

[8]  Valentin Jijkoun,et al.  Retrieving answers from frequently asked questions pages on the web , 2005, CIKM '05.

[9]  Adam L. Berger,et al.  A Maximum Entropy Approach to Natural Language Processing , 1996, CL.

[10]  W. Bruce Croft,et al.  Finding similar questions in large question and answer archives , 2005, CIKM '05.

[11]  Shih-Hung Wu,et al.  An integrated knowledge-based and machine learning approach for Chinese question classification , 2005, 2005 International Conference on Natural Language Processing and Knowledge Engineering.

[12]  Noriko Tomuro,et al.  Interrogative Reformulation Patterns and Acquisition of Question Paraphrases , 2003, IWP@ACL.

[13]  A. M. Turing,et al.  Computing Machinery and Intelligence , 1950, The Philosophy of Artificial Intelligence.

[14]  Chong-Ho Choi,et al.  Input feature selection for classification problems , 2002, IEEE Trans. Neural Networks.

[15]  Guy Lapalme,et al.  The QUANTUM Question Answering System , 2001, TREC.

[16]  Dell Zhang,et al.  Question classification using support vector machines , 2003, SIGIR.