Embedding Time Differences in Context-sensitive Neural Networks for Learning Time to Event

We propose an effective context-sensitive neural model for the task of time to event (TTE) prediction, which aims to predict the amount of time to/from the occurrence of given events in streaming content. We investigate this problem in the context of a multi-task learning framework, which we enrich with time difference embeddings. To conduct this research, we develop a multi-genre dataset of English events about soccer competitions and academy awards ceremonies, as well as their relevant tweets obtained from Twitter. Our model is 1.4 and 3.3 hours more accurate than the current state-of-the-art model in estimating TTE on English and Dutch tweets respectively. We examine different aspects of our model to illustrate its source of improvement.1

[1]  Michael Gertz,et al.  Multilingual and cross-domain temporal tagging , 2012, Language Resources and Evaluation.

[2]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[3]  Ellen Riloff,et al.  Distinguishing Past, On-going, and Future Events: The EventStatus Corpus , 2016, EMNLP.

[4]  Iryna Gurevych,et al.  Temporal Anchoring of Events for the TimeBank Corpus , 2016, ACL.

[5]  Katja Markert,et al.  Dataset Reproducibility and IR Methods in Timeline Summarization , 2020, LREC.

[6]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[7]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[8]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[9]  Roberto Cipolla,et al.  Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  H. Zeliger.  Predicting disease onset in clinically healthy people , 2016, Interdisciplinary toxicology.

[11]  Jane S. Paulsen,et al.  A new model for prediction of the age of onset and penetrance for Huntington's disease based on CAG length , 2004, Clinical genetics.

[12]  Dan Roth,et al.  Temporal Common Sense Acquisition with Minimal Supervision , 2020, ACL.

[13]  Michael Gertz,et al.  Temporal Tagging on Different Domains: Challenges, Strategies, and Gold Standards , 2012, LREC.

[14]  Antal van den Bosch,et al.  Estimating Time to Event of Future Events Based on Linguistic Cues on Twitter , 2018 .

[15]  Antal van den Bosch,et al.  Estimating Time to Event from Tweets Using Temporal Expressions , 2014 .

[16]  Hiroya Takamura,et al.  Numeracy-600K: Learning Numeracy for Detecting Exaggerated Information in Market Comments , 2019, ACL.

[17]  Walter Daelemans,et al.  Evaluating Unsupervised Dutch Word Embeddings as a Linguistic Resource , 2016, LREC.

[18]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[19]  Sameer Singh,et al.  Do NLP Models Know Numbers? Probing Numeracy in Embeddings , 2019, EMNLP.

[20]  Iryna Gurevych,et al.  Event Time Extraction with a Decision Tree of Neural Classifiers , 2018, TACL.