Narrative Modeling with Memory Chains and Semantic Supervision

Story comprehension requires a deep semantic understanding of the narrative, making it a challenging task. Inspired by previous studies on ROC Story Cloze Test, we propose a novel method, tracking various semantic aspects with external neural memory chains while encouraging each to focus on a particular semantic aspect. Evaluated on the task of story ending prediction, our model demonstrates superior performance to a collection of competitive baselines, setting a new state of the art.

[1]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[2]  Chris Dyer,et al.  On the State of the Art of Evaluation in Neural Language Models , 2017, ICLR.

[3]  Yevgeniy Puzikov,et al.  LSDSem 2017: Exploring Data Generation Methods for the Story Cloze Test , 2017, LSDSem@EACL.

[4]  Bing Liu,et al.  Opinion observer: analyzing and comparing opinions on the Web , 2005, WWW '05.

[5]  Yejin Choi,et al.  The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task , 2017, CoNLL.

[6]  Eugene Charniak,et al.  Toward a model of children's story comprehension , 1972 .

[7]  Chung Hee Hwang,et al.  Episodic Logic Meets Little Red Riding Hood: A Comprehensive, Natural Representation for Language Un , 2000 .

[8]  Nathanael Chambers,et al.  A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories , 2016, NAACL.

[9]  Dan Roth,et al.  Story Comprehension for Predicting What Happens Next , 2017, EMNLP.

[10]  Stephanie W. Haas.  The Creative Process: A Computer Model of Storytelling and Creativity, by Scott R. Turner , 1996, J. Am. Soc. Inf. Sci..

[11]  Noah A. Smith,et al.  Probabilistic Frame-Semantic Parsing , 2010, NAACL.

[12]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[13]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[14]  Todor Mihaylov,et al.  Story Cloze Ending Selection Baselines and Data Examination , 2017, LSDSem@EACL.

[15]  Dan Roth,et al.  Two Discourse Driven Language Models for Semantics , 2016, ACL.

[16]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Nathanael Chambers,et al.  LSDSem 2017 Shared Task: The Story Cloze Test , 2017, LSDSem@EACL.

[18]  Martin Wattenberg,et al.  Ad click prediction: a view from the trenches , 2013, KDD.

[19]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[20]  Terry Winograd,et al.  Understanding natural language , 1974 .

[21]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[22]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[23]  Jason Weston,et al.  Tracking the World State with Recurrent Entity Networks , 2016, ICLR.