Semantic Annotation of Textual Entailment

We introduce a new formal semantic model for annotating textual entailments, that describes restrictive, intersective and appositive modification. The model contains a formally defined interpreted lexicon, which specifies the inventory of symbols and the supported semantic operators, and an informally defined annotation scheme that instructs annotators in which way to bind words and constructions from a given pair of premise and hypothesis to the interpreted lexicon. We explore the applicability of the proposed model to the Recognizing Textual Entailment (RTE) 1-4 corpora and describe a first-stage annotation scheme based on which manual annotation work was carried out. The constructions we annotated were found to occur in 80.65% of the entailments in RTE 1-4 and were annotated with cross-annotator agreement of 68% on average. The annotated RTE corpora are publicly available for the research community.

[1]  Christopher D. Manning,et al.  Natural Logic for Textual Inference , 2007, ACL-PASCAL@ACL.

[2]  Roy Bar-Haim,et al.  The Second PASCAL Recognising Textual Entailment Challenge , 2006 .

[3]  Ido Dagan,et al.  A Confidence Model for Syntactically-Motivated Entailment Proofs , 2011, RANLP.

[4]  J Quinonero Candela,et al.  Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment , 2006, Lecture Notes in Computer Science.

[5]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[6]  Johan Bos,et al.  Recognising Textual Entailment with Logical Inference , 2005, HLT.

[7]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[8]  Stephen Pulman,et al.  Using the Framework , 1996 .

[9]  Konstantina Garoufi.  Towards a Better Understanding of Applied Textual Entailment: Annotation and Evaluation of the RTE-2 Dataset , 2007 .

[10]  Ido Dagan,et al.  The Sixth PASCAL Recognizing Textual Entailment Challenge , 2009, TAC.

[11]  Ido Dagan,et al.  Semantic Annotation for Textual Entailment Recognition , 2012, MICAI.

[12]  Peter Clark,et al.  The Seventh PASCAL Recognizing Textual Entailment Challenge , 2011, TAC.

[13]  Kalina Bontcheva,et al.  Text Processing with GATE , 2011 .

[14]  Victor Manual Sánchez Valencia,et al.  Studies on natural logic and categorial grammar , 1991 .

[15]  Lawrence S. Moss,et al.  LOGICS FOR THE RELATIONAL SYLLOGISTIC , 2008, The Review of Symbolic Logic.

[16]  Mark Steedman,et al.  Wide-Coverage Semantic Representations from a CCG Parser , 2004, COLING.

[17]  Cordelia Schmid,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[18]  Ido Dagan,et al.  The Fourth PASCAL Recognizing Textual Entailment Challenge | NIST , 2009 .