An Improved Neural Baseline for Temporal Relation Extraction

Determining temporal relations (e.g., before or after) between events has been a challenging natural language understanding task, partly due to the difficulty to generate large amounts of high-quality training data. Consequently, neural approaches have not been widely used on it, or showed only moderate improvements. This paper proposes a new neural system that achieves about 10% absolute improvement in accuracy over the previous best system (25% error reduction) on two benchmark datasets. The proposed system is trained on the state-of-the-art MATRES dataset and applies contextualized word embeddings, a Siamese encoder of a temporal common sense knowledge base, and global inference via integer linear programming (ILP). We suggest that the new approach could serve as a strong baseline for future research in this area.

[1]  Chen Lin,et al.  Temporal Annotation in the Clinical Domain , 2014, TACL.

[2]  Marie-Francine Moens,et al.  Temporal Information Extraction by Predicting Relative Time-lines , 2018, EMNLP.

[3]  Taylor Cassidy,et al.  An Annotation Framework for Dense Event Ordering , 2014, ACL.

[4]  James H. Martin,et al.  Timelines from Text: Identification of Syntactic Temporal Relations , 2007, International Conference on Semantic Computing (ICSC 2007).

[5]  Tommaso Caselli,et al.  SemEval-2010 Task 13: TempEval-2 , 2010, *SEMEVAL.

[6]  Dong Wang,et al.  Relation Classification via Recurrent Neural Network , 2015, ArXiv.

[7]  Yusuke Miyao,et al.  Classifying Temporal Relations by Bidirectional LSTM over Dependency Paths , 2017, ACL.

[8]  Chen Lin,et al.  Representations of Time Expressions for Temporal Relation Extraction with Convolutional Neural Networks , 2017, BioNLP.

[9]  James Joseph Biundo,et al.  Analysis of Contingency Tables , 1969 .

[10]  James Pustejovsky,et al.  Temporal Processing with the TARSQI Toolkit , 2008, COLING.

[11]  Chen Lin,et al.  Neural Temporal Relation Extraction , 2017, EACL.

[12]  Dan Roth,et al.  A Structured Learning Approach to Temporal Relation Extraction , 2017, EMNLP.

[13]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[14]  James Pustejovsky,et al.  SemEval-2007 Task 15: TempEval Temporal Relation Identification , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[15]  Chris Callison-Burch,et al.  Magnitude: A Fast, Efficient Universal Vector Embedding Utility Package , 2018, EMNLP.

[16]  James Pustejovsky,et al.  SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and Temporal Relations , 2013, *SEMEVAL.

[17]  Nathanael Chambers,et al.  CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures , 2016, EVENTS@HLT-NAACL.

[18]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[19]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[20]  Eneko Agirre,et al.  SemEval-2015 Task 4: TimeLine: Cross-Document Event Ordering , 2015, *SEMEVAL.

[21]  Dan Roth,et al.  CogCompTime: A Tool for Understanding Time in Natural Language , 2018, EMNLP.

[22]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[23]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[24]  Dan Roth,et al.  Joint Inference for Event Timeline Construction , 2012, EMNLP.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[27]  James Pustejovsky,et al.  SemEval-2015 Task 6: Clinical TempEval , 2015, *SEMEVAL.

[28]  Marie-Francine Moens,et al.  Structured Learning for Temporal Relation Extraction from Clinical Records , 2017, EACL.

[29]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[30]  Shan Wang,et al.  Classifying Temporal Relations Between Events , 2007, ACL.

[31]  Jun Zhao,et al.  Relation Classification via Convolutional Deep Neural Network , 2014, COLING.

[32]  Hao Wu,et al.  Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource , 2018, NAACL.

[33]  Yuji Matsumoto,et al.  Jointly Identifying Temporal Relations with Markov Logic , 2009, ACL.

[34]  Weihong Deng,et al.  Recurrent convolutional neural network for video classification , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[35]  Ming Yang,et al.  Bidirectional Long Short-Term Memory Networks for Relation Classification , 2015, PACLIC.

[36]  Nathanael Chambers,et al.  Unsupervised Learning of Narrative Event Chains , 2008, ACL.

[37]  Thomas G. Dietterich.  Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms , 1998, Neural Computation.

[38]  Nathanael Chambers,et al.  Jointly Combining Implicit Constraints Improves Temporal Ordering , 2008, EMNLP.

[39]  Olivier Ferret,et al.  Neural Architecture for Temporal Relation Extraction: A Bi-LSTM Approach for Detecting Narrative Containers , 2017, ACL.

[40]  Anna Rumshisky,et al.  Context-Aware Neural Model for Temporal Information Extraction , 2018, ACL.

[41]  James Pustejovsky,et al.  SemEval-2017 Task 12: Clinical TempEval , 2017, *SEMEVAL.

[42]  James Pustejovsky,et al.  SemEval-2015 Task 5: QA TempEval - Evaluating Temporal Information Understanding with Question Answering , 2015, *SEMEVAL.

[43]  Hao Wu,et al.  Joint Reasoning for Temporal and Causal Relations , 2018, ACL.

[44]  Martha Palmer,et al.  Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation , 2016 .

[45]  Hao Wu,et al.  A Multi-Axis Annotation Scheme for Event Temporal Relations , 2018, ACL.

[46]  James Pustejovsky,et al.  Machine Learning of Temporal Relations , 2006, ACL.

[47]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.