Effective Distant Supervision for Temporal Relation Extraction

A principal barrier to training temporal relation extraction models in new domains is the lack of varied, high quality examples and the challenge of collecting more. We present a method of automatically collecting distantly-supervised examples of temporal relations. We scrape and automatically label event pairs where the temporal relations are made explicit in text, then mask out those explicit cues, forcing a model trained on this data to learn other signals. We demonstrate that a pre-trained Transformer model is able to transfer from the weakly labeled examples to human-annotated benchmarks in both zero-shot and few-shot settings, and that the masking scheme is important in improving generalization.

[1]  Nathanael Chambers,et al.  Conditional Generation of Temporally-ordered Event Sequences , 2021, ACL/IJCNLP.

[2]  Christopher De Sa,et al.  Data Programming: Creating Large Training Sets, Quickly , 2016, NIPS.

[3]  Patrick Pantel,et al.  VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations , 2004, EMNLP.

[4]  Dan Roth,et al.  Temporal Common Sense Acquisition with Minimal Supervision , 2020, ACL.

[5]  Quoc V. Le,et al.  ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators , 2020, ICLR.

[6]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[7]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[8]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[9]  Monica S. Lam,et al.  AutoQA: From Databases to Q&A Semantic Parsers with Only Synthetic Training Data , 2020, EMNLP.

[10]  Wenlin Yao,et al.  Temporal Event Knowledge Acquisition via Identifying Narratives , 2018, ACL.

[11]  Noah D. Goodman,et al.  DisSent: Learning Sentence Representations from Explicit Discourse Relations , 2019, ACL.

[12]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[13]  Hao Wu,et al.  Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource , 2018, NAACL.

[14]  Tanya Goyal,et al.  Embedding time expressions for deep temporal ordering models , 2019, ACL.

[15]  Christopher Ré,et al.  Training Classifiers with Natural Language Explanations , 2018, ACL.

[16]  Samuel R. Bowman,et al.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference , 2017, NAACL.

[17]  Hao Wu,et al.  A Multi-Axis Annotation Scheme for Event Temporal Relations , 2018, ACL.

[18]  Taylor Cassidy,et al.  An Annotation Framework for Dense Event Ordering , 2014, ACL.

[19]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[20]  Jacob Andreas,et al.  Unnatural Language Processing: Bridging the Gap Between Synthetic and Natural Language Data , 2020, ArXiv.

[21]  Benjamin Van Durme,et al.  Fine-Grained Temporal Relation Extraction , 2019, ACL.

[22]  Thomas Wolf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[23]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[24]  Nanyun Peng,et al.  Joint Event and Temporal Relation Extraction with Shared Representations and Structured Prediction , 2019, EMNLP.

[25]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.