Semantic Annotation for Textual Entailment Recognition

We introduce a new semantic annotation scheme for the Recognizing Textual Entailment (RTE) dataset as well as a manually annotated dataset that uses this scheme. The scheme addresses three types of modification that license entailment patterns: restrictive, appositive and conjunctive, with a formal semantic specification of these patterns' contribution for establishing entailment. These inferential constructions were found to occur in 77.68% of the entailments in the RTE 1-3 corpora. They were annotated with cross-annotator agreement of 70.73% on average. A central aim of our annotations is to examine components that address these phenomena in RTE systems. Specifically, the new annotated dataset is used for examining a syntactic rule base within the BIUTEE recognizer, a publicly available entailment system. According to our tests, the rule base is rarely used to process the phenomena annotated in our corpus and most of the recognition work is done by other components in the system.

[1]  J. McCawley.  Parentheticals and discontinuous constituent structure , 1982 .

[2]  Jan Svartvik,et al.  A __ comprehensive grammar of the English language , 1988 .

[3]  P. Sells.  Restrictive and non-restrictive modification , 1985 .

[4]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[5]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[6]  Dekang Lin,et al.  Automatic Retrieval and Clustering of Similar Words , 1998, ACL.

[7]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[8]  Patrick Pantel,et al.  DIRT @SBT@discovery of inference rules from text , 2001, KDD '01.

[9]  Mark Steedman,et al.  Example Selection for Bootstrapping Statistical Parsers , 2003, NAACL.

[10]  Nizar Habash,et al.  A Categorial Variation Database for English , 2003, NAACL.

[11]  Patrick Pantel,et al.  VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations , 2004, EMNLP.

[12]  Rodolfo Delmonte.  VENSES - A Linguistically-Based System for Semantic Evaluation , 2005, Proces. del Leng. Natural.

[13]  L. Ferro,et al.  MITRE ’ s Submissions to the EU Pascal RTE Challenge , 2005 .

[14]  Andrew Hickl,et al.  Recognizing Textual Entailment with LCC’s G ROUNDHOG System , 2005 .

[15]  Enrique Alfonseca,et al.  Application of the Bleu algorithm for recognising textual entailments , 2005 .

[16]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[17]  Cordelia Schmid,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[18]  Roy Bar-Haim,et al.  The Second PASCAL Recognising Textual Entailment Challenge , 2006 .

[19]  Alessandro Moschitti,et al.  Learning textual entailment from examples , 2006 .

[20]  Dan I. Moldovan,et al.  COGEX at the Second Recognizing Textual Entailment Challenge , 2006 .

[21]  Dominic Abrams,et al.  Language, Speech, and Communication , 2006 .

[22]  J Quinonero Candela,et al.  Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment , 2006, Lecture Notes in Computer Science.

[23]  Francesca Del Gobbo.  On the syntax and semantics of appositive relative clauses , 2007 .

[24]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[25]  Andrew Hickl,et al.  A Discourse Commitment-Based Framework for Recognizing Textual Entailment , 2007, ACL-PASCAL@ACL.

[26]  Dan I. Moldovan,et al.  COGEX at RTE 3 , 2007, ACL-PASCAL@ACL.

[27]  Ido Dagan,et al.  Semantic Inference at the Lexical-Syntactic Level for Textual Entailment Recognition , 2007, ACL-PASCAL@ACL.

[28]  Adrian Iftene,et al.  Hypothesis Transformation and Semantic Variability Rules Used in Recognizing Textual Entailment , 2007, ACL-PASCAL@ACL.

[29]  Josef van Genabith,et al.  Coling 2008: Proceedings of the workshop on Cross-Framework and Cross-Domain Parser Evaluation , 2008, COLING 2008.

[30]  Ido Dagan,et al.  The Fourth PASCAL Recognizing Textual Entailment Challenge , 2008, TAC.

[31]  Christopher D. Manning,et al.  The Stanford Typed Dependencies Representation , 2008, CF+CDPE@COLING.

[32]  Ido Dagan,et al.  The Fourth PASCAL Recognizing Textual Entailment Challenge | NIST , 2009 .

[33]  Ido Dagan,et al.  The Sixth PASCAL Recognizing Textual Entailment Challenge , 2009, TAC.

[34]  Ido Dagan,et al.  Building Textual Entailment Specialized Data Sets: a Methodology for Isolating Linguistic Phenomena Relevant to Inference , 2010, LREC.

[35]  Dan Roth,et al.  “Ask Not What Textual Entailment Can Do for You...” , 2010, ACL.

[36]  Fredric C. Gey,et al.  Proceedings of LREC , 2010 .

[37]  Idan Szpektor,et al.  Textual Entailment , 2010, NAACL.

[38]  Peter Clark,et al.  The Seventh PASCAL Recognizing Textual Entailment Challenge , 2011, TAC.

[39]  Ido Dagan,et al.  A Confidence Model for Syntactically-Motivated Entailment Proofs , 2011, RANLP.

[40]  Kalina Bontcheva,et al.  Text Processing with GATE , 2011 .

[41]  A Syntax-based Rule-base for Textual Entailment and a Semantic Truth Value Annotator , 2012 .

[42]  Graeme Hirst,et al.  Recognizing Textual Entailment , 2012 .