SWIM: A Simple Word Interaction Model for Implicit Discourse Relation Recognition

Capturing the semantic interaction of pairs of words across arguments and proper argument representation are both crucial issues in implicit discourse relation recognition. The current state-of-the-art represents arguments as distributional vectors that are computed via bi-directional Long Short-Term Memory networks (BiLSTMs), known to have significant model complexity. In contrast, we demonstrate that word-weighted averaging can encode argument representation which can be incorporated with word pair information efficiently. By saving an order of magnitude in parameters and eschewing the recurrent structure, our proposed model achieves equivalent performance, but trains seven times faster.

[1]  Yang Liu,et al.  Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention , 2016, EMNLP.

[2]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[3]  Dinh Phung,et al.  Journal of Machine Learning Research: Preface , 2014 .

[4]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[5]  Kevin Gimpel,et al.  Towards Universal Paraphrastic Sentence Embeddings , 2015, ICLR.

[6]  Jian Su,et al.  Predicting Discourse Connectives for Implicit Discourse Relation Recognition , 2010, COLING.

[7]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[8]  Yang Liu,et al.  Implicit Discourse Relation Classification via Multi-Task Neural Networks , 2016, AAAI.

[9]  Xuanjing Huang,et al.  Implicit Discourse Relation Detection via a Deep Architecture with Gated Relevance Network , 2016, ACL.

[10]  Treebank Penn,et al.  Linguistic Data Consortium , 1999 .

[11]  Min-Yen Kan,et al.  Using Discourse Signals for Robust Instructor Intervention Prediction , 2017, AAAI.

[12]  Zheng-Yu Niu,et al.  Leveraging Synthetic Discourse Data via Multi-task Learning for Implicit Discourse Relation Recognition , 2013, ACL.

[13]  Ani Nenkova,et al.  Automatic sense prediction for implicit discourse relations in text , 2009, ACL.

[14]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[15]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[16]  Nianwen Xue,et al.  Discovering Implicit Discourse Relations Through Brown Cluster Pair Representation and Coreference Patterns , 2014, EACL.

[17]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[18]  Nianwen Xue,et al.  Improving the Inference of Implicit Discourse Relations via Classifying Explicit Discourse Connectives , 2015, NAACL.

[19]  Hwee Tou Ng,et al.  Recognizing Implicit Discourse Relations in the Penn Discourse Treebank , 2009, EMNLP.

[20]  N. Jojic,et al.  Ieee Transactions on Signal Processing: Supplement on Secure Media 1 Facecerts Ieee Transactions on Signal Processing: Supplement on Secure Media 2 , 2003 .

[21]  Hal Daumé,et al.  Deep Unordered Composition Rivals Syntactic Methods for Text Classification , 2015, ACL.

[22]  Danushka Bollegala,et al.  A Semi-Supervised Approach to Improve Classification of Infrequent Discourse Relations Using Feature Vector Extension , 2010, EMNLP.

[23]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[24]  Rongrong Ji,et al.  Variational Neural Discourse Relation Recognizer , 2016, EMNLP.

[25]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.