Enhanced Story Representation by ConceptNet for Predicting Story Endings

Predicting endings for narrative stories is a grand challenge for machine commonsense reasoning. The task requires ac- curate representation of the story semantics and structured logic knowledge. Pre-trained language models, such as BERT, made progress recently in this task by exploiting spurious statistical patterns in the test dataset, instead of 'understanding' the stories per se. In this paper, we propose to improve the representation of stories by first simplifying the sentences to some key concepts and second modeling the latent relation- ship between the key ideas within the story. Such enhanced sentence representation, when used with pre-trained language models, makes substantial gains in prediction accuracy on the popular Story Cloze Test without utilizing the biased validation data.

[1]  Jun Zhao,et al.  Conditional Generative Adversarial Networks for Commonsense Machine Comprehension , 2017, IJCAI.

[2]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[3]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[4]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[5]  James R. Meehan,et al.  TALE-SPIN, An Interactive Program that Writes Stories , 1977, IJCAI.

[6]  Naoya Inoue,et al.  An RNN-based Binary Classifier for the Story Cloze Test , 2017, LSDSem@EACL.

[7]  Minlie Huang,et al.  Story Ending Generation with Incremental Encoding and Commonsense Knowledge , 2018, AAAI.

[8]  Zhou Yu,et al.  Incorporating Structured Commonsense Knowledge in Story Completion , 2018, AAAI.

[9]  Nathanael Chambers,et al.  A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories , 2016, NAACL.

[10]  Ting Liu,et al.  Story Ending Prediction by Transferable BERT , 2019, IJCAI.

[11]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[12]  Mark O. Riedl,et al.  A Simple and Effective Approach to the Story Cloze Test , 2018, NAACL-HLT.

[13]  Jin-Mao Wei,et al.  A Multi-Attention based Neural Network with External Knowledge for Story Ending Predicting Task , 2018, COLING.

[14]  Yejin Choi,et al.  ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning , 2019, AAAI.

[15]  Dan Roth,et al.  A Joint Model for Semantic Sequences: Frames, Entities, Sentiments , 2017, CoNLL.