TE4AV: Textual Entailment for Answer Validation

The textual entailment (TE) task consists of discovering unidirectional semantic inferences between the meanings of two text snippets. Taking advantage of this, in this paper we propose using the TE system as an answer validation (AV) engine to improve the performance of question answering (QA) systems and help humans in the assessment of QA systems' outputs. To achieve these aims and in order to assess the overall performance of our TE system and its application in QA tasks, two evaluation environments are presented: pure entailment and QA-response evaluation. The former uses the corpus and methodology of the PASCAL recognizing textual entailment challenges, whereas for the latter we use the data provided by the answer validation exercise competition within the cross-language evaluation forum. The system, the evaluations environments and the experiments developed are discussed throughout the paper.

[1]  Óscar Ferrández,et al.  On the Application of Lexical-Syntactic Knowledge to the Answer Validation Exercise , 2007, CLEF.

[2]  Dan Roth,et al.  Semantic and Logical Inference Model for Textual Entailment , 2007, ACL-PASCAL@ACL.

[3]  Karen Spärck Jones.  A statistical interpretation of term specificity and its application in retrieval , 2021, J. Documentation.

[4]  Patrick Pantel,et al.  DIRT @SBT@discovery of inference rules from text , 2001, KDD '01.

[5]  Scott Settembre.  Textual Entailment Using Univariate Density Model and Maximizing Discriminant Function , 2007, ACL-PASCAL@ACL.

[6]  Emiel Krahmer,et al.  Dependency-based paraphrasing for recognizing textual entailment , 2007, ACL-PASCAL@ACL.

[7]  Catherine Blake,et al.  The Role of Sentence Structure in Recognizing Textual Entailment , 2007, ACL-PASCAL@ACL.

[8]  Bernardo Magnini,et al.  Detecting Expected Answer Relations through Textual Entailment , 2008, CICLing.

[9]  Matthew A. Jaro,et al.  Probabilistic linkage of large public health data files. , 1995, Statistics in medicine.

[10]  Günter Neumann,et al.  Recognizing Textual Entailment Using Sentence Similarity based on Dependency Tree Skeletons , 2007, ACL-PASCAL@ACL.

[11]  Sanda M. Harabagiu,et al.  Methods for Using Textual Entailment in Open-Domain Question Answering , 2006, ACL.

[12]  Zornitsa Kozareva,et al.  Combining data-driven systems for improving Named Entity Recognition , 2005, Data Knowl. Eng..

[13]  Ido Dagan,et al.  Semantic Inference at the Lexical-Syntactic Level for Textual Entailment Recognition , 2007, ACL-PASCAL@ACL.

[14]  David W. Conrath,et al.  Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy , 1997, ROCLING/IJCLCLP.

[15]  Sanda M. Harabagiu,et al.  Textual Entailment Through Extended Lexical Overlap and Lexico-Semantic Matching , 2007, ACL-PASCAL@ACL.

[16]  Dekang Lin,et al.  DIRT – Discovery of Inference Rules from Text , 2001 .

[17]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[18]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .

[19]  Patrick Pantel,et al.  VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations , 2004, EMNLP.

[20]  Carol Peters.  What happened in CLEF 2008: Introduction to the Working Notes , 2004, CLEF.

[21]  Ion Androutsopoulos,et al.  Learning Textual Entailment using SVMs and String Similarity Measures , 2007, ACL-PASCAL@ACL.

[22]  Dan I. Moldovan,et al.  COGEX at RTE 3 , 2007, ACL-PASCAL@ACL.

[23]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[24]  Óscar Ferrández,et al.  A Perspective-Based Approach for Solving Textual Entailment Recognition , 2007, ACL-PASCAL@ACL.

[25]  Dekang Lin,et al.  Dependency-Based Evaluation of Minipar , 2003 .

[26]  Carol Peters.  What Happened in CLEF 2007 , 2007, CLEF.

[27]  Adam Janin,et al.  Mutaphrase: Paraphrasing with FrameNet , 2007, ACL-PASCAL@ACL.

[28]  Stefan Thater,et al.  A Semantic Approach To Textual Entailment: System Evaluation and Task Analysis , 2007, ACL-PASCAL@ACL.

[29]  M. Felisa Verdejo,et al.  UNED at Answer Validation Exercise 2007 , 2007, CLEF.

[30]  M. Felisa Verdejo,et al.  Overview of the Answer Validation Exercise 2007 , 2007, CLEF.

[31]  Philip Resnik,et al.  Using Information Content to Evaluate Semantic Similarity in a Taxonomy , 1995, IJCAI.

[32]  Milen Kouylekov.  Recognizing Textual Entailment with Tree Edit Distance: Application to Question Answering and Information Extraction , 2006 .

[33]  Adrian Iftene,et al.  Hypothesis Transformation and Semantic Variability Rules Used in Recognizing Textual Entailment , 2007, ACL-PASCAL@ACL.

[34]  Daniel G. Bobrow,et al.  Precision-focused Textual Inference , 2007, ACL-PASCAL@ACL.

[35]  Karen Sparck Jones.  A statistical interpretation of term specificity and its application in retrieval , 1972 .

[36]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[37]  Ido Dagan,et al.  Scaling Web-based Acquisition of Entailment Relations , 2004, EMNLP.

[38]  Dekang Lin,et al.  An Information-Theoretic Definition of Similarity , 1998, ICML.

[39]  Andrew Hickl,et al.  A Discourse Commitment-Based Framework for Recognizing Textual Entailment , 2007, ACL-PASCAL@ACL.

[40]  Óscar Ferrández,et al.  DLSITE-1: Lexical Analysis for Solving Textual Entailment Recognition , 2007, NLDB.

[41]  M S Waterman,et al.  Identification of common molecular subsequences. , 1981, Journal of molecular biology.