Memory Augmented Attention Model for Chinese Implicit Discourse Relation Recognition

Recently, Chinese implicit discourse relation recognition has attracted more and more attention, since it is crucial to understand the Chinese discourse text. In this paper, we propose a novel memory augmented attention model which represents the arguments using an attention-based neural network and preserves the crucial information with an external memory network which captures each discourse relation clustering structure to support the relation inference. Extensive experiments demonstrate that our proposed model can achieve the new state-of-the-art results on Chinese Discourse Treebank. We further leverage network visualization to show why our attention and memory model are effective.

[1]  Pascal Denis,et al.  Comparing Word Representations for Implicit Discourse Relation Classification , 2015, EMNLP.

[2]  Yang Liu,et al.  Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention , 2016, EMNLP.

[3]  J. Elman.  Distributed Representations, Simple Recurrent Networks, And Grammatical Structure , 1991 .

[4]  Xuan Zhang,et al.  The Virginia Tech System at CoNLL-2016 Shared Task on Shallow Discourse Parsing , 2016, CoNLL Shared Task.

[5]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[6]  Haoran Li,et al.  An End-to-End Chinese Discourse Parser with Adaptation to Explicit and Non-explicit Relation Recognition , 2016, CoNLL Shared Task.

[7]  Hwee Tou Ng,et al.  A PDTB-styled end-to-end discourse parser , 2012, Natural Language Engineering.

[8]  Ani Nenkova,et al.  Using Syntax to Disambiguate Explicit Discourse Connectives in Text , 2009, ACL.

[9]  Fang Kong,et al.  Building Chinese Discourse Corpus with Connective-driven Dependency Tree Structure , 2014, EMNLP.

[10]  Man Lan,et al.  Two End-to-end Shallow Discourse Parsers for English and Chinese in CoNLL-2016 Shared Task , 2016, CoNLL.

[11]  Nianwen Xue,et al.  A Systematic Study of Neural Discourse Models for Implicit Discourse Relation , 2017, EACL.

[12]  Jiajun Zhang,et al.  Implicit Discourse Relation Recognition for English and Chinese with Multiview Modeling and Effective Representation Learning , 2017, ACM Trans. Asian Low Resour. Lang. Inf. Process..

[13]  Tu Me.  Automatically Parsing Chinese Discourse Based on Maximum Entropy , 2014 .

[14]  Pascal Denis,et al.  Learning Connective-based Word Representations for Implicit Discourse Relation Identification , 2016, EMNLP.

[15]  Eduard H. Hovy,et al.  Recursive Deep Models for Discourse Parsing , 2014, EMNLP.

[16]  Richard Socher,et al.  Ask Me Anything: Dynamic Memory Networks for Natural Language Processing , 2015, ICML.

[17]  Hwee Tou Ng,et al.  Recognizing Implicit Discourse Relations in the Penn Discourse Treebank , 2009, EMNLP.

[18]  Nianwen Xue,et al.  Discovering Implicit Discourse Relations Through Brown Cluster Pair Representation and Coreference Patterns , 2014, EACL.

[19]  Jacob Eisenstein,et al.  Representation Learning for Text-level Discourse Parsing , 2014, ACL.

[20]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[21]  Yu Zhou,et al.  Predicting Implicit Discourse Relation with Multi-view Modeling and Effective Representation Learning , 2016, NLPCC/ICCPOL.

[22]  Christian Chiarcos,et al.  Do We Really Need All Those Rich Linguistic Features? A Neural Network-Based Approach to Implicit Sense Labeling , 2016, CoNLL.

[23]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[24]  Yuping Zhou,et al.  The Chinese Discourse TreeBank: a Chinese corpus annotated with discourse relations , 2015, Lang. Resour. Evaluation.

[25]  Hwee Tou Ng,et al.  CoNLL 2016 Shared Task on Multilingual Shallow Discourse Parsing , 2016, CoNLL.

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[27]  Marko Bajec,et al.  Discourse Sense Classification from Scratch using Focused RNNs , 2016, CoNLL Shared Task.

[28]  Jacob Eisenstein,et al.  One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations , 2014, TACL.

[29]  Nianwen Xue,et al.  Annotating Discourse Connectives in the Chinese Treebank , 2005, FCA@ACL.

[30]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[31]  Nianwen Xue,et al.  Neural Network Models for Implicit Discourse Relation Classification in English and Chinese without Surface Features , 2016, ArXiv.

[32]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[33]  Yuping Zhou,et al.  PDTB-style Discourse Annotation of Chinese Text , 2012, ACL.