A Recurrent Neural Model with Attention for the Recognition of Chinese Implicit Discourse Relations

We introduce an attention-based Bi-LSTM for Chinese implicit discourse relations and demonstrate that modeling argument pairs as a joint sequence can outperform word order-agnostic approaches. Our model benefits from a partial sampling scheme and is conceptually simple, yet achieves state-of-the-art performance on the Chinese Discourse Treebank. We also visualize its attention activity to illustrate the model's ability to selectively focus on the relevant parts of an input sequence.

[1]  Yang Liu,et al.  Recognizing Implicit Discourse Relations via Repeated Reading: Neural Networks with Multi-Level Attention , 2016, EMNLP.

[2]  Nathanael Chambers,et al.  A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories , 2016, NAACL.

[3]  Phil Blunsom,et al.  Teaching Machines to Read and Comprehend , 2015, NIPS.

[4]  Graeme Hirst,et al.  Text-level Discourse Parsing with Rich Linguistic Features , 2012, ACL.

[5]  Fatemeh Torabi Asr,et al.  Uniform Information Density at the Level of Discourse Relations: Negation Markers and Discourse Connective Omission , 2015 .

[6]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[7]  Gholamreza Haffari,et al.  A Latent Variable Recurrent Neural Network for Discourse Relation Language Models , 2016, ArXiv.

[8]  Alex Lascarides,et al.  Temporal interpretation, discourse relations and commonsense entailment , 1993, The Language of Time - A Reader.

[9]  Joyce Yue Chai,et al.  Discourse processing for context question answering based on linguistic knowledge , 2007, Knowl. Based Syst..

[10]  William C. Mann,et al.  Rhetorical Structure Theory: Toward a functional theory of text organization , 1988 .

[11]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[12]  Hsin-Hsi Chen,et al.  Chinese Discourse Relation Recognition , 2011, IJCNLP.

[13]  Christian Chiarcos,et al.  Do We Really Need All Those Rich Linguistic Features? A Neural Network-Based Approach to Implicit Sense Labeling , 2016, CoNLL.

[14]  Xuanjing Huang,et al.  Implicit Discourse Relation Detection via a Deep Architecture with Gated Relevance Network , 2016, ACL.

[15]  Hai Zhao,et al.  Shallow Discourse Parsing Using Convolutional Neural Network , 2016, CoNLL.

[16]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[17]  Bowen Zhou,et al.  Applying deep learning to answer selection: A study and an open task , 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

[18]  Pengcheng Zhang,et al.  Discourse Relation Sense Classification Systems for CoNLL-2016 Shared Task , 2016, CoNLL Shared Task.

[19]  Bonnie L. Webber,et al.  D-LTAG: extending lexicalized TAG to discourse , 2004, Cogn. Sci..

[20]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[21]  Rashmi Prasad,et al.  The Penn Discourse Treebank , 2004, LREC.

[22]  Masaaki Nagata,et al.  Single-Document Summarization as a Tree Knapsack Problem , 2013, EMNLP.

[23]  Yaojie Lu,et al.  Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition , 2015, EMNLP.

[24]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Man Lan,et al.  Two End-to-end Shallow Discourse Parsers for English and Chinese in CoNLL-2016 Shared Task , 2016, CoNLL.

[27]  Andrew Hickl,et al.  Using Discourse Commitments to Recognize Textual Entailment , 2008, COLING.

[28]  Hwee Tou Ng,et al.  CoNLL 2016 Shared Task on Multilingual Shallow Discourse Parsing , 2016, CoNLL.

[29]  Ani Nenkova,et al.  Automatic sense prediction for implicit discourse relations in text , 2009, ACL.

[30]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[31]  Marko Bajec,et al.  Discourse Sense Classification from Scratch using Focused RNNs , 2016, CoNLL Shared Task.

[32]  William S. Horton,et al.  Why or what next? Eye movements reveal expectations about discourse direction , 2010 .

[33]  Jacob Eisenstein,et al.  Discourse Connectors for Latent Subjectivity in Sentiment Analysis , 2013, NAACL.

[34]  Yuping Zhou,et al.  PDTB-style Discourse Annotation of Chinese Text , 2012, ACL.

[35]  Nianwen Xue,et al.  Robust Non-Explicit Neural Discourse Parser in English and Chinese , 2016, CoNLL Shared Task.