Fine-grained Temporal Relation Extraction with Ordered-Neuron LSTM and Graph Convolutional Networks

Fine-grained temporal relation extraction (FineTempRel) aims to recognize the durations and timeline of event mentions in text. A missing part in the current deep learning models for FineTempRel is their failure to exploit the syntactic structures of the input sentences to enrich the representation vectors. In this work, we propose to fill this gap by introducing novel methods to integrate the syntactic structures into the deep learning models for FineTempRel. The proposed model focuses on two types of syntactic information from the dependency trees, i.e., the syntax-based importance scores for representation learning of the words and the syntactic connections to identify important context words for the event mentions. We also present two novel techniques to facilitate the knowledge transfer between the subtasks of FineTempRel, leading to a novel model with the state-of-the-art performance for this task.

[1]  Dan Roth,et al.  An Improved Neural Baseline for Temporal Relation Extraction , 2019, EMNLP.

[2]  Aaron C. Courville,et al.  Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks , 2018, ICLR.

[3]  Anna Rumshisky,et al.  Context-Aware Neural Model for Temporal Information Extraction , 2018, ACL.

[4]  Olivier Ferret,et al.  Neural Architecture for Temporal Relation Extraction: A Bi-LSTM Approach for Detecting Narrative Containers , 2017, ACL.

[5]  Thien Huu Nguyen,et al.  Cross-Task Instance Representation Interactions and Label Dependencies for Joint Information Extraction with Graph Convolutional Networks , 2021, NAACL.

[6]  Jing Lu,et al.  Joint Inference for Event Coreference Resolution , 2016, COLING.

[7]  Jerry R. Hobbs,et al.  Modeling and Learning Vague Event Durations for Temporal Reasoning , 2007, AAAI.

[8]  Chen Lin,et al.  Neural Temporal Relation Extraction , 2017, EACL.

[9]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[10]  Dan Roth,et al.  Exploiting Partially Annotated Data for Temporal Relation Extraction , 2018, *SEM@NAACL-HLT.

[11]  Jennifer Williams,et al.  Extracting and modeling durations for habits and events from Twitter , 2012, ACL.

[12]  Paramita Mirza,et al.  CATENA: CAusal and TEmporal relation extraction from NAtural language texts , 2016, COLING.

[13]  Steven Bethard,et al.  ClearTK-TimeML: A minimalist approach to TempEval 2013 , 2013, *SEMEVAL.

[14]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[15]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[16]  Yubo Chen,et al.  Knowledge Enhanced Event Causality Identification with Mention Masking Generalizations , 2020, IJCAI.

[17]  Ralph Grishman,et al.  New York University 2016 System for KBP Event Nugget: A Deep Learning Approach , 2016, TAC.

[18]  Yusuke Miyao,et al.  Classifying Temporal Relations by Bidirectional LSTM over Dependency Paths , 2017, ACL.

[19]  Nanyun Peng,et al.  Deep Structured Neural Network for Event Temporal Relation Extraction , 2019, CoNLL.

[20]  Chen Lin,et al.  Multilayered temporal modeling for the clinical domain , 2016, J. Am. Medical Informatics Assoc..

[21]  Martha Palmer,et al.  Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation , 2016 .

[22]  Amir Pouran Ben Veyseh,et al.  Graph Transformer Networks with Syntactic and Semantic Structures for Event Argument Extraction , 2020, FINDINGS.

[23]  Hao Wu,et al.  A Multi-Axis Annotation Scheme for Event Temporal Relations , 2018, ACL.

[24]  Nathanael Chambers,et al.  Using Query Patterns to Learn the Duration of Events , 2011, IWCS.

[25]  Taylor Cassidy,et al.  An Annotation Framework for Dense Event Ordering , 2014, ACL.

[26]  Benjamin Van Durme,et al.  Fine-Grained Temporal Relation Extraction , 2019, ACL.

[27]  James Pustejovsky,et al.  Machine Learning of Temporal Relations , 2006, ACL.

[28]  Nanyun Peng,et al.  Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction , 2019, EMNLP.

[29]  Franck Dernoncourt,et al.  Exploiting the Syntax-Model Consistency for Neural Relation Extraction , 2020, ACL.

[30]  Thien Huu Nguyen,et al.  Graph Convolutional Networks for Event Causality Identification with Rich Document-level Structures , 2021, NAACL.

[31]  Chantal van Son,et al.  MEANTIME, the NewsReader Multilingual Event and Time Corpus , 2016, LREC.

[32]  Ralph Grishman,et al.  Graph Convolutional Networks With Argument-Aware Pooling for Event Detection , 2018, AAAI.

[33]  Vincent Ng,et al.  Classifying Temporal Relations with Rich Linguistic Knowledge , 2013, NAACL.

[34]  Thien Huu Nguyen,et al.  Exploiting Document Structures and Cluster Consistencies for Event Coreference Resolution , 2021, ACL/IJCNLP.

[35]  Hao Wu,et al.  Joint Reasoning for Temporal and Causal Relations , 2018, ACL.

[36]  Tanya Goyal,et al.  Embedding time expressions for deep temporal ordering models , 2019, ACL.

[37]  Heng Ji,et al.  Building a Cross-document Event-Event Relation Corpus , 2016, LAW@ACL.

[38]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[39]  Dan Roth,et al.  A Structured Learning Approach to Temporal Relation Extraction , 2017, EMNLP.

[40]  Jing Lu,et al.  Joint Learning for Event Coreference Resolution , 2017, ACL.