Exploiting Partially Annotated Data for Temporal Relation Extraction

Annotating temporal relations (TempRel) between events described in natural language is known to be labor intensive, partly because the total number of TempRels is quadratic in the number of events. As a result, only a small number of documents are typically annotated, limiting the coverage of various lexical/semantic phenomena. In order to improve existing approaches, one possibility is to make use of the readily available, partially annotated data (P as in partial) that cover more documents. However, missing annotations in P are known to hurt, rather than help, existing systems. This work is a case study in exploring various usages of P for TempRel extraction. Results show that despite missing annotations, P is still a useful supervision signal for this task within a constrained bootstrapping learning framework. The system described in this system is publicly available.

[1]  Hao Wu,et al.  Joint Reasoning for Temporal and Causal Relations , 2018, ACL.

[2]  Dan Roth,et al.  A Structured Learning Approach to Temporal Relation Extraction , 2017, EMNLP.

[3]  Takashi Chikayama,et al.  UTTime: Temporal Relation Classification using Deep Syntactic Features , 2013, *SEMEVAL.

[4]  James Pustejovsky,et al.  SemEval-2007 Task 15: TempEval Temporal Relation Identification , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[5]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[6]  Regina Barzilay,et al.  Inducing Temporal Graphs , 2006, EMNLP.

[7]  Nate Chambers.  NavyTime: Event and Time Ordering from Raw Text , 2013, SemEval@NAACL-HLT.

[8]  James Pustejovsky,et al.  SemEval-2017 Task 12: Clinical TempEval , 2017, *SEMEVAL.

[9]  Martha Palmer,et al.  Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation , 2016 .

[10]  Hao Wu,et al.  A Multi-Axis Annotation Scheme for Event Temporal Relations , 2018, ACL.

[11]  Dan Roth,et al.  A Linear Programming Formulation for Global Inference in Natural Language Tasks , 2004, CoNLL.

[12]  James Pustejovsky,et al.  Machine Learning of Temporal Relations , 2006, ACL.

[13]  James Pustejovsky,et al.  SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and Temporal Relations , 2013, *SEMEVAL.

[14]  Nathanael Chambers,et al.  CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures , 2016, EVENTS@HLT-NAACL.

[15]  Dan Roth,et al.  Joint Inference for Event Timeline Construction , 2012, EMNLP.

[16]  Paramita Mirza,et al.  CATENA: CAusal and TEmporal relation extraction from NAtural language texts , 2016, COLING.

[17]  Taylor Cassidy,et al.  An Annotation Framework for Dense Event Ordering , 2014, ACL.

[18]  James H. Martin,et al.  Timelines from Text: Identification of Syntactic Temporal Relations , 2007, International Conference on Semantic Computing (ICSC 2007).

[19]  Dan Roth,et al.  Incidental Supervision: Moving beyond Supervised Learning , 2017, AAAI.

[20]  Hao Wu,et al.  Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource , 2018, NAACL.

[21]  Nathanael Chambers,et al.  Jointly Combining Implicit Constraints Improves Temporal Ordering , 2008, EMNLP.

[22]  Steven Bethard,et al.  ClearTK-TimeML: A minimalist approach to TempEval 2013 , 2013, *SEMEVAL.

[23]  Pascal Denis,et al.  Predicting Globally-Coherent Temporal Structures from Texts via Endpoint Inference and Graph Decomposition , 2011, IJCAI.

[24]  Eneko Agirre,et al.  SemEval-2015 Task 4: TimeLine: Cross-Document Event Ordering , 2015, *SEMEVAL.

[25]  Shan Wang,et al.  Classifying Temporal Relations Between Events , 2007, ACL.

[26]  Tommaso Caselli,et al.  SemEval-2010 Task 13: TempEval-2 , 2010, *SEMEVAL.

[27]  James Pustejovsky,et al.  Temporal Processing with the TARSQI Toolkit , 2008, COLING.

[28]  Ming-Wei Chang,et al.  Structured learning with constrained conditional models , 2012, Machine Learning.