Automatic Feature Engineering for Answer Selection and Extraction

This paper proposes a framework for automatically engineering features for two important tasks of question answering: answer sentence selection and answer extraction. We represent question and answer sentence pairs with linguistic structures enriched by semantic information, where the latter is produced by automatic classifiers, e.g., question classifier and Named Entity Recognizer. Tree kernels applied to such structures enable a simple way to generate highly discriminative structural features that combine syntactic and semantic information encoded in the input trees. We conduct experiments on a public benchmark from TREC to compare with previous systems for answer sentence selection and answer extraction. The results show that our models greatly improve on the state of the art, e.g., up to 22% on F1 (relative improvement) for answer extraction, while using no additional resources and no manual feature engineering.

[1]  Jennifer Chu-Carroll,et al.  Building Watson: An Overview of the DeepQA Project , 2010, AI Mag..

[2]  Christopher D. Manning,et al.  Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering , 2010, COLING.

[3]  Alessandro Moschitti,et al.  Automatic Learning of Textual Entailments with Cross-Pair Similarities , 2006, ACL.

[4]  Alessandro Moschitti,et al.  Kernels on Linguistic Structures for Answer Extraction , 2008, ACL.

[5]  Dekang Lin,et al.  A Probabilistic Answer Type Model , 2006, EACL.

[6]  Noah A. Smith,et al.  What is the Jeopardy Model? A Quasi-Synchronous Grammar for QA , 2007, EMNLP.

[7]  Dan Roth,et al.  Learning Question Classifiers , 2002, COLING.

[8]  Doug Downey,et al.  Local and Global Algorithms for Disambiguation to Wikipedia , 2011, ACL.

[9]  John M. Prager,et al.  Open-Domain Question-Answering , 2007, Found. Trends Inf. Retr..

[10]  Alessandro Moschitti,et al.  Syntactic and Semantic Kernels for Short Text Pair Categorization , 2009, EACL.

[11]  Alessandro Moschitti,et al.  Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees , 2006, ECML.

[12]  Silvia Quarteroni,et al.  Evaluating Multi-focus Natural Language Queries over Data Services , 2012, LREC.

[13]  Alessandro Moschitti,et al.  Kernel methods, syntax and semantics for relational text categorization , 2008, CIKM '08.

[14]  Hamish Cunningham,et al.  Identification of the Question Focus: Combining Syntactic Analysis and Ontology-based Lookup through the User Interaction , 2010, LREC.

[15]  Alessandro Moschitti,et al.  Efficient Graph Kernels for Textual Entailment Recognition , 2011, Fundam. Informaticae.

[16]  Alessandro Moschitti,et al.  A machine learning approach to textual entailment recognition , 2009, Natural Language Engineering.

[17]  Alessandro Moschitti,et al.  Learning Adaptable Patterns for Passage Reranking , 2013, CoNLL.

[18]  Alessandro Moschitti,et al.  Structural relationships for large-scale learning of answer re-ranking , 2012, SIGIR '12.

[19]  Michael Collins,et al.  New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron , 2002, ACL.

[20]  Noah A. Smith,et al.  Tree Edit Models for Recognizing Textual Entailments, Paraphrases, and Answers to Questions , 2010, NAACL.

[21]  Alessandro Moschitti,et al.  Building structures from classifiers for passage reranking , 2013, CIKM.

[22]  Chris Callison-Burch,et al.  Answer Extraction as Sequence Tagging with Tree Edit Distance , 2013, NAACL.

[23]  Alessandro Moschitti,et al.  Learning Semantic Textual Similarity with Structural Representations , 2013, ACL.