Temporal Reasoning in Natural Language Inference

We introduce five new natural language inference (NLI) datasets focused on temporal reasoning. We recast four existing datasets annotated for event duration—how long an event lasts—and event ordering—how events are temporally arranged—into more than one million NLI examples. We use these datasets to investigate how well neural models trained on a popular NLI corpus capture these forms of temporal reasoning.

[1]  James F. Allen.  Towards a General Theory of Action and Time , 1984, Artif. Intell..

[2]  Martha Palmer,et al.  Richer Event Description: Integrating event coreference with temporal, causal and bridging annotation , 2016 .

[3]  Hao Wu,et al.  A Multi-Axis Annotation Scheme for Event Temporal Relations , 2018, ACL.

[4]  Michael Rovatsos.  Events , 1952, Journal of Failure Analysis and Prevention.

[5]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[6]  Samuel R. Bowman,et al.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference , 2017, NAACL.

[7]  Kevin Duh,et al.  Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework , 2017, IJCNLP.

[8]  Christopher Potts,et al.  A large annotated corpus for learning natural language inference , 2015, EMNLP.

[9]  Anna Rumshisky,et al.  Context-Aware Neural Model for Temporal Information Extraction , 2018, ACL.

[10]  Olivier Ferret,et al.  Neural Architecture for Temporal Relation Extraction: A Bi-LSTM Approach for Detecting Narrative Containers , 2017, ACL.

[11]  Benjamin Van Durme,et al.  Fine-Grained Temporal Relation Extraction , 2019, ACL.

[12]  Rachel Rudinger,et al.  Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation , 2018, BlackboxNLP@EMNLP.

[13]  Ellie Pavlick,et al.  How well do NLI models capture verb veridicality? , 2019, EMNLP.

[14]  Holger Schwenk,et al.  Supervised Learning of Universal Sentence Representations from Natural Language Inference Data , 2017, EMNLP.

[15]  Nate Blaylock,et al.  Building Timelines from Narrative Clinical Records: Initial Results Based-on Deep Natural Language Understanding , 2011, BioNLP@ACL.

[16]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[17]  Christopher D. Manning,et al.  Stanza: A Python Natural Language Processing Toolkit for Many Human Languages , 2020, ACL.

[18]  庄于雨.  Going on Vacation , 2010 .

[19]  Drew McDermott,et al.  A Temporal Logic for Reasoning About Processes and Plans , 1982, Cogn. Sci..

[20]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[21]  Dan Roth,et al.  A Structured Learning Approach to Temporal Relation Extraction , 2017, EMNLP.

[22]  Bertram C. Bruce.  A Model for Temporal References and Its Application in a Question Answering Program , 1972, Artif. Intell..

[23]  Aoying Zhou,et al.  Event phase oriented news summarization , 2017, World Wide Web.

[24]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[25]  Hal Daumé,et al.  Deep Unordered Composition Rivals Syntactic Methods for Text Classification , 2015, ACL.

[26]  Van Durme,et al.  Extracting implicit knowledge from text , 2009 .

[27]  Dan Roth,et al.  “Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding , 2019, EMNLP.

[28]  Bhavana Dalvi,et al.  Reasoning about Actions and State Changes by Injecting Commonsense Knowledge , 2018, EMNLP.

[29]  James Pustejovsky,et al.  SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and Temporal Relations , 2013, *SEMEVAL.

[30]  Nathanael Chambers,et al.  CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures , 2016, EVENTS@HLT-NAACL.

[31]  G. P. Bhattacharjee,et al.  Temporal representation and reasoning in artificial intelligence: A review , 2001 .

[32]  R'emi Louf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[33]  Yonatan Belinkov,et al.  Don’t Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference , 2019, ACL.

[34]  Alexander Nakhimovsky,et al.  Temporal Reasoning in Natural Language Understanding: The Temporal Structure of the Narrative , 1987, EACL.

[35]  Sheng Zhang,et al.  Ordinal Common-sense Inference , 2016, TACL.

[36]  Taylor Cassidy,et al.  Dense Event Ordering with a Multi-Pass Architecture , 2014, TACL.

[37]  John McCarthy,et al.  SOME PHILOSOPHICAL PROBLEMS FROM THE STANDPOINT OF ARTI CIAL INTELLIGENCE , 1987 .

[38]  Nanyun Peng,et al.  TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions , 2020, EMNLP.

[39]  Jeffrey M. Zacks,et al.  Event structure in perception and conception. , 2001, Psychological bulletin.

[40]  Nanyun Peng,et al.  Deep Structured Neural Network for Event Temporal Relation Extraction , 2019, CoNLL.

[41]  Adina Williams,et al.  Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition , 2020, ACL.

[42]  Carolyn Penstein Rosé,et al.  TDDiscourse: A Dataset for Discourse-Level Temporal Ordering of Events , 2019, SIGdial.

[43]  Nianwen Xue,et al.  Temporal relation discovery between events and temporal expressions identified in clinical narrative , 2013, J. Biomed. Informatics.

[44]  Lenhart K. Schubert.  Can we derive general world knowledge from texts , 2002 .

[45]  Rui Yan,et al.  Natural Language Inference by Tree-Based Convolution and Heuristic Matching , 2015, ACL.

[46]  James Pustejovsky,et al.  SemEval-2015 Task 5: QA TempEval - Evaluating Temporal Information Understanding with Question Answering , 2015, *SEMEVAL.

[47]  Hao Wu,et al.  Joint Reasoning for Temporal and Causal Relations , 2018, ACL.

[48]  Iryna Gurevych,et al.  Temporal Anchoring of Events for the TimeBank Corpus , 2016, ACL.

[49]  Dan Roth,et al.  Temporal Common Sense Acquisition with Minimal Supervision , 2020, ACL.

[50]  Kenneth M. Kahn,et al.  Mechanizing Temporal Knowledge , 1977, Artif. Intell..

[51]  Stephen Pulman,et al.  Using the Framework , 1996 .