Memorizing All for Implicit Discourse Relation Recognition

Implicit discourse relation recognition is a challenging task due to the absence of the necessary informative clue from explicit connectives. The prediction of relations requires a deep understanding of the semantic meanings of sentence pairs. As implicit discourse relation recognizer has to carefully tackle the semantic similarity of the given sentence pairs and the severe data sparsity issue exists in the meantime, it is supposed to be beneficial from mastering the entire training data. Thus in this paper, we propose a novel memory mechanism to tackle the challenges for further performance improvement. The memory mechanism is adequately memorizing information by pairing representations and discourse relations of all training instances, which right fills the slot of the data-hungry issue in the current implicit discourse relation recognizer. Our experiments show that our full model with memorizing the entire training set reaches new state-of-the-art against strong baselines, which especially for the first time exceeds the milestone of 60% accuracy in the 4-way task.

[1]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[2]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[3]  Jürgen Schmidhuber,et al.  Training Very Deep Networks , 2015, NIPS.

[4]  Ruihong Huang,et al.  Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph , 2018, NAACL.

[5]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[6]  Christian Chiarcos,et al.  Do We Really Need All Those Rich Linguistic Features? A Neural Network-Based Approach to Implicit Sense Labeling , 2016, CoNLL.

[7]  Nianwen Xue,et al.  A Systematic Study of Neural Discourse Models for Implicit Discourse Relation , 2017, EACL.

[8]  Gholamreza Haffari,et al.  A Latent Variable Recurrent Neural Network for Discourse Relation Language Models , 2016, ArXiv.

[9]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[10]  Giuseppe Carenini,et al.  Abstractive Summarization of Product Reviews Using Discourse Structure , 2014, EMNLP.

[11]  Zheng-Yu Niu,et al.  Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification , 2017, EMNLP.

[12]  Nianwen Xue,et al.  Improving the Inference of Implicit Discourse Relations via Classifying Explicit Discourse Connectives , 2015, NAACL.

[13]  Min-Yen Kan,et al.  Linguistic Properties Matter for Implicit Discourse Relation Recognition: Combining Semantic Interaction, Topic Continuity and Attribution , 2018, AAAI.

[14]  Yaojie Lu,et al.  Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition , 2015, EMNLP.

[15]  Jacob Eisenstein,et al.  One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations , 2014, TACL.

[16]  Hai Zhao,et al.  A Stacking Gated Neural Architecture for Implicit Discourse Relation Classification , 2016, EMNLP.

[17]  Yang Liu,et al.  Implicit Discourse Relation Classification via Multi-Task Neural Networks , 2016, AAAI.

[18]  Hai Zhao,et al.  Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification , 2017, ACL.

[19]  Xuanjing Huang,et al.  Discourse Relations Detection via a Mixed Generative-Discriminative Framework , 2016, AAAI.

[20]  Hwee Tou Ng,et al.  CoNLL 2016 Shared Task on Multilingual Shallow Discourse Parsing , 2016, CoNLL.

[21]  Min Zhang,et al.  Using active learning to expand training data for implicit discourse relation recognition , 2018, EMNLP.

[22]  Xuanjing Huang,et al.  Implicit Discourse Relation Detection via a Deep Architecture with Gated Relevance Network , 2016, ACL.

[23]  Hai Zhao,et al.  A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware? , 2018, COLING.

[24]  Hwee Tou Ng,et al.  Recognizing Implicit Discourse Relations in the Penn Discourse Treebank , 2009, EMNLP.

[25]  Peter Jansen,et al.  Discourse Complements Lexical Semantics for Non-factoid Answer Reranking , 2014, ACL.

[26]  Yugo Murawaki,et al.  A Knowledge-Augmented Neural Network Model for Implicit Discourse Relation Classification , 2018, COLING.

[27]  Hai Zhao,et al.  Shallow Discourse Parsing Using Convolutional Neural Network , 2016, CoNLL.

[28]  Yann Dauphin,et al.  Language Modeling with Gated Convolutional Networks , 2016, ICML.

[29]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[30]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[31]  Pascal Denis,et al.  Comparing Word Representations for Implicit Discourse Relation Classification , 2015, EMNLP.

[32]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[33]  Min-Yen Kan,et al.  SWIM: A Simple Word Interaction Model for Implicit Discourse Relation Recognition , 2017, IJCAI.

[34]  Yang Liu,et al.  Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention , 2016, EMNLP.

[35]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[36]  Jian Su,et al.  Predicting Discourse Connectives for Implicit Discourse Relation Recognition , 2010, COLING.

[37]  Hwee Tou Ng,et al.  The CoNLL-2015 Shared Task on Shallow Discourse Parsing , 2015, CoNLL.

[38]  Hai Zhao,et al.  Dependency or Span, End-to-End Uniform Semantic Role Labeling , 2019, AAAI.

[39]  Hai Zhao,et al.  Deep Enhanced Representation for Implicit Discourse Relation Recognition , 2018, COLING.

[40]  Hai Zhao,et al.  Implicit Discourse Relation Recognition with Context-aware Character-enhanced Embeddings , 2016, COLING.

[41]  Christian Chiarcos,et al.  A Recurrent Neural Model with Attention for the Recognition of Chinese Implicit Discourse Relations , 2017, ACL.

[42]  Jianwu Dang,et al.  Implicit Discourse Relation Recognition using Neural Tensor Network with Interactive Attention and Sparse Learning , 2018, COLING.

[43]  Ani Nenkova,et al.  Automatic sense prediction for implicit discourse relations in text , 2009, ACL.