Let’s do it “again”: A First Computational Approach to Detecting Adverbial Presupposition Triggers

We introduce the task of predicting adverbial presupposition triggers such as also and again. Solving such a task requires detecting recurring or similar events in the discourse context, and has applications in natural language generation tasks such as summarization and dialogue systems. We create two new datasets for the task, derived from the Penn Treebank and the Annotated English Gigaword corpora, as well as a novel attention mechanism tailored to this task. Our attention mechanism augments a baseline recurrent neural network without the need for additional trainable parameters, minimizing the added computational cost of our mechanism. We demonstrate that our model statistically outperforms a number of baselines, including an LSTM-based language model.

[1]  C. R.,et al.  On referring , 1950 .

[2]  Kang Qiang,et al.  The Use of Too as a Pragmatic Presupposition Trigger , 2012 .

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[5]  Robert Stalnaker,et al.  Presuppositions , 1973, J. Philos. Log..

[6]  E. B. Coleman,et al.  A Measure of Information Gained during Prose Learning , 1968 .

[7]  Chris Dyer,et al.  On the State of the Art of Evaluation in Neural Language Models , 2017, ICLR.

[8]  Javad Zare,et al.  Presupposition Trigger-A Comparative Analysis of Broadcast News Discourse , 2012 .

[9]  G. Frege.  Über Sinn und Bedeutung , 1892 .

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[12]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[13]  Ting Liu,et al.  Attention-over-Attention Neural Networks for Reading Comprehension , 2016, ACL.

[14]  Ellen Riloff,et al.  A Rule-based Question Answering System for Reading Comprehension Tests , 2000 .

[15]  Dan Klein,et al.  A Generative Constituent-Context Model for Improved Grammar Induction , 2002, ACL.

[16]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[17]  Richard Socher,et al.  A Deep Reinforced Model for Abstractive Summarization , 2017, ICLR.

[18]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[19]  Wilson L. Taylor,et al.  “Cloze Procedure”: A New Tool for Measuring Readability , 1953 .

[20]  Jason Weston,et al.  The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations , 2015, ICLR.

[21]  Earl F. Rankin,et al.  Comparable Cloze and Multiple-Choice Comprehension Test Scores. , 1969 .

[22]  Henk Zeevat,et al.  Optimality Theory and Pragmatics , 2003 .

[23]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[24]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[25]  Robert Stalnaker,et al.  On the Representation of Context , 1996, J. Log. Lang. Inf..

[26]  David Kaplan,et al.  What is Russell’s Theory of Descriptions? , 1970 .

[27]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[28]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[29]  Layth Muthana Khaleel.  An Analysis of Presupposition Triggers in English Journalistic Texts , 2010 .

[30]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[31]  Jackie Chi Kit Cheung,et al.  Capturing Pragmatic Knowledge in Article Usage Prediction using LSTMs , 2016, COLING.

[32]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.