Topic Tensor Network for Implicit Discourse Relation Recognition in Chinese

In the literature, most of the previous studies on English implicit discourse relation recognition only use sentence-level representations, which cannot provide enough semantic information in Chinese due to its unique paratactic characteristics. In this paper, we propose a topic tensor network to recognize Chinese implicit discourse relations with both sentence-level and topic-level representations. In particular, besides encoding arguments (discourse units) using a gated convolutional network to obtain sentence-level representations, we train a simplified topic model to infer the latent topic-level representations. Moreover, we feed the two pairs of representations to two factored tensor networks, respectively, to capture both the sentence-level interactions and topic-level relevance using multi-slice tensors. Experimentation on CDTB, a Chinese discourse corpus, shows that our proposed model significantly outperforms several state-of-the-art baselines in both micro and macro F1-scores.

[1]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[2]  Min Zhang,et al.  Using active learning to expand training data for implicit discourse relation recognition , 2018, EMNLP.

[3]  Phil Blunsom,et al.  Neural Variational Inference for Text Processing , 2015, ICML.

[4]  Zheng-Yu Niu,et al.  Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification , 2017, EMNLP.

[5]  Houfeng Wang,et al.  A Two-Stage Parsing Method for Text-Level Discourse Analysis , 2017, ACL.

[6]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[7]  G. Meade.  Building a Discourse-Tagged Corpus in the Framework of Rhetorical Structure Theory , 2001 .

[8]  Jing Li,et al.  Topic Memory Networks for Short Text Classification , 2018, EMNLP.

[9]  Hai Zhao,et al.  Deep Enhanced Representation for Implicit Discourse Relation Recognition , 2018, COLING.

[10]  Xuanjing Huang,et al.  Implicit Discourse Relation Detection via a Deep Architecture with Gated Relevance Network , 2016, ACL.

[11]  Leila Kosseim,et al.  Attention for Implicit Discourse Relation Recognition , 2018, LREC.

[12]  Fang Kong,et al.  Building Chinese Discourse Corpus with Connective-driven Dependency Tree Structure , 2014, EMNLP.

[13]  Christian Chiarcos,et al.  A Recurrent Neural Model with Attention for the Recognition of Chinese Implicit Discourse Relations , 2017, ACL.

[14]  Yann Dauphin,et al.  Language Modeling with Gated Convolutional Networks , 2016, ICML.

[15]  Jian Su,et al.  Predicting Discourse Connectives for Implicit Discourse Relation Recognition , 2010, COLING.

[16]  Yaojie Lu,et al.  Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition , 2015, EMNLP.

[17]  Nicolas Le Roux,et al.  A latent factor model for highly multi-relational data , 2012, NIPS.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jianwu Dang,et al.  Implicit Discourse Relation Recognition using Neural Tensor Network with Interactive Attention and Sparse Learning , 2018, COLING.

[20]  Yang Liu,et al.  Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention , 2016, EMNLP.

[21]  Graeme Hirst,et al.  A Linear-Time Bottom-Up Discourse Parser with Constraints and Post-Editing , 2014, ACL.

[22]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[23]  Yang Liu,et al.  Memory Augmented Attention Model for Chinese Implicit Discourse Relation Recognition , 2017, CCL.

[24]  Yuping Zhou,et al.  The Chinese Discourse TreeBank: a Chinese corpus annotated with discourse relations , 2015, Lang. Resour. Evaluation.

[25]  Hwee Tou Ng,et al.  CoNLL 2016 Shared Task on Multilingual Shallow Discourse Parsing , 2016, CoNLL.

[26]  Ani Nenkova,et al.  Automatic sense prediction for implicit discourse relations in text , 2009, ACL.

[27]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[28]  Wei Shi,et al.  Using Explicit Discourse Connectives in Translation for Implicit Discourse Relation Classification , 2017, IJCNLP.

[29]  Yang Liu,et al.  Implicit Discourse Relation Classification via Multi-Task Neural Networks , 2016, AAAI.

[30]  Min-Yen Kan,et al.  Linguistic Properties Matter for Implicit Discourse Relation Recognition: Combining Semantic Interaction, Topic Continuity and Attribution , 2018, AAAI.

[31]  Fei Xia,et al.  The Penn Chinese TreeBank: Phrase structure annotation of a large corpus , 2005, Natural Language Engineering.

[32]  Fang Kong,et al.  A CDT-Styled End-to-End Chinese Discourse Parser , 2017, NLPCC/ICCPOL.

[33]  Hwee Tou Ng,et al.  Recognizing Implicit Discourse Relations in the Penn Discourse Treebank , 2009, EMNLP.

[34]  Baobao Chang,et al.  Max-Margin Tensor Neural Network for Chinese Word Segmentation , 2014, ACL.

[35]  Hai Zhao,et al.  A Stacking Gated Neural Architecture for Implicit Discourse Relation Classification , 2016, EMNLP.

[36]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[37]  Hai Zhao,et al.  Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification , 2017, ACL.