A Structured Learning Approach to Temporal Relation Extraction

Identifying temporal relations between events is an essential step towards natural language understanding. However, the temporal relation between two events in a story depends on, and is often dictated by, relations among other events. Consequently, effectively identifying temporal relations between events is a challenging problem even for human annotators. This paper suggests that it is important to take these dependencies into account while learning to identify these relations and proposes a structured learning approach to address this challenge. As a byproduct, this provides a new perspective on handling missing relations, a known issue that hurts existing methods. As we show, the proposed approach results in significant improvements on the two commonly used data sets for this problem.

[1]  Nathanael Chambers,et al.  Jointly Combining Implicit Constraints Improves Temporal Ordering , 2008, EMNLP.

[2]  James F. Allen,et al.  Temporal Evaluation , 2011, ACL.

[3]  Ben Wellner,et al.  Three Approaches to Learning TLINKs in TimeML , 2007 .

[4]  Tommaso Caselli,et al.  SemEval-2010 Task 13: TempEval-2 , 2010, *SEMEVAL.

[5]  Ming-Wei Chang,et al.  Guiding Semi-Supervision with Constraint-Driven Learning , 2007, ACL.

[6]  Dan Roth,et al.  Joint Inference for Event Timeline Construction , 2012, EMNLP.

[7]  Dan Roth,et al.  Extraction of events and temporal expressions from clinical narratives , 2013, J. Biomed. Informatics.

[8]  James Pustejovsky,et al.  Temporal Processing with the TARSQI Toolkit , 2008, COLING.

[9]  Ming-Wei Chang,et al.  Structured learning with constrained conditional models , 2012, Machine Learning.

[10]  Michael Gertz,et al.  HeidelTime: High Quality Rule-Based Extraction and Normalization of Temporal Expressions , 2010, *SEMEVAL.

[11]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[12]  Mihai Surdeanu.  Overview of the TAC2013 Knowledge Base Population Evaluation: English Slot Filling and Temporal Slot Filling , 2013, TAC.

[13]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[14]  Steven Bethard,et al.  ClearTK-TimeML: A minimalist approach to TempEval 2013 , 2013, *SEMEVAL.

[15]  Eneko Agirre,et al.  SemEval-2015 Task 4: TimeLine: Cross-Document Event Ordering , 2015, *SEMEVAL.

[16]  Dan Roth,et al.  Learning and Inference over Constrained Output , 2005, IJCAI.

[17]  Angel X. Chang,et al.  SUTime: A library for recognizing and normalizing time expressions , 2012, LREC.

[18]  Luke S. Zettlemoyer,et al.  Context-dependent Semantic Parsing for Time Expressions , 2014, ACL.

[19]  James Joseph Biundo,et al.  Analysis of Contingency Tables , 1969 .

[20]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT' 98.

[21]  James H. Martin,et al.  Timelines from Text: Identification of Syntactic Temporal Relations , 2007, International Conference on Semantic Computing (ICSC 2007).

[22]  Ming-Wei Chang,et al.  Structured Output Learning with Indirect Supervision , 2010, ICML.

[23]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[24]  Marie-Francine Moens,et al.  Structured Learning for Temporal Relation Extraction from Clinical Records , 2017, EACL.

[25]  James Pustejovsky,et al.  SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and Temporal Relations , 2013, *SEMEVAL.

[26]  Shan Wang,et al.  Classifying Temporal Relations Between Events , 2007, ACL.

[27]  James Pustejovsky,et al.  Machine Learning of Temporal Relations , 2006, ACL.

[28]  Dan Roth,et al.  A Linear Programming Formulation for Global Inference in Natural Language Tasks , 2004, CoNLL.

[29]  James Pustejovsky,et al.  TimeML: Robust Specification of Event and Temporal Expressions in Text , 2003, New Directions in Question Answering.

[30]  Dan Roth,et al.  A Robust Shallow Temporal Reasoning System , 2012, HLT-NAACL.

[31]  James Pustejovsky,et al.  SemEval-2017 Task 12: Clinical TempEval , 2017, *SEMEVAL.

[32]  James Pustejovsky,et al.  SemEval-2007 Task 15: TempEval Temporal Relation Identification , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[33]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[34]  Regina Barzilay,et al.  Inducing Temporal Graphs , 2006, EMNLP.

[35]  Dan Roth,et al.  Learning Based Java for Rapid Development of NLP Systems , 2010, LREC.

[36]  Heng Ji,et al.  Tackling representation, annotation and classification challenges for temporal knowledge base population , 2014, Knowledge and Information Systems.

[37]  Nate Chambers.  NavyTime: Event and Time Ordering from Raw Text , 2013, SemEval@NAACL-HLT.

[38]  James F. Allen.  Towards a General Theory of Action and Time , 1984, Artif. Intell..

[39]  Takashi Chikayama,et al.  UTTime: Temporal Relation Classification using Deep Syntactic Features , 2013, *SEMEVAL.

[40]  James Pustejovsky,et al.  SemEval-2015 Task 5: QA TempEval - Evaluating Temporal Information Understanding with Question Answering , 2015, *SEMEVAL.

[41]  James H. Martin,et al.  CU-TMP: Temporal Relation Classification Using Syntactic and Semantic Features , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[42]  Taylor Cassidy,et al.  An Annotation Framework for Dense Event Ordering , 2014, ACL.