Building Context-aware Clause Representations for Situation Entity Type Classification

Capabilities to categorize a clause based on the type of situation entity (e.g., events, states and generic statements) the clause introduces to the discourse can benefit many NLP applications. Observing that the situation entity type of a clause depends on discourse functions the clause plays in a paragraph and the interpretation of discourse functions depends heavily on paragraph-wide contexts, we propose to build context-aware clause representations for predicting situation entity types of clauses. Specifically, we propose a hierarchical recurrent neural network model to read a whole paragraph at a time and jointly learn representations for all the clauses in the paragraph by extensively modeling context influences and inter-dependencies of clauses. Experimental results show that our model achieves the state-of-the-art performance for clause-level situation entity classification on the genre-rich MASC+Wiki corpus, which approaches human-level performance.

[1]  Yue Lu,et al.  A Sequence Labeling Convolutional Network and Its Application to Handwritten String Recognition , 2017, IJCAI.

[2]  S. Roumyana.  ASPECTUAL ENTITIES AND TENSE IN DISCOURSE , 2002 .

[3]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[4]  A. Koller,et al.  Speech Acts: An Essay in the Philosophy of Language , 1969 .

[5]  Alexis Palmer,et al.  Situation Entity Annotation , 2014, LAW@COLING.

[6]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[7]  Daniel Jurafsky,et al.  A Hierarchical Neural Autoencoder for Paragraphs and Documents , 2015, ACL.

[8]  Manfred Pinkal,et al.  Situation entity types: automatic classification of clause-level aspect , 2016, ACL.

[9]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[10]  Eric Nichols,et al.  Named Entity Recognition with Bidirectional LSTM-CNNs , 2015, TACL.

[11]  Holger Schwenk,et al.  Supervised Learning of Universal Sentence Representations from Natural Language Inference Data , 2017, EMNLP.

[12]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[13]  Kathleen McKeown,et al.  Learning Methods to Combine Linguistic Indicators:Improving Aspectual Classification and Revealing Linguistic Insights , 2000, CL.

[14]  Manfred Pinkal,et al.  Automatic recognition of habituals: a three-way classification of clausal aspect , 2015, EMNLP.

[15]  Ashish Vaswani,et al.  Supertagging With LSTMs , 2016, NAACL.

[16]  Daniel Marcu,et al.  Sentence Level Discourse Parsing using Syntactic and Lexical Information , 2003, NAACL.

[17]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[18]  Nils Reiter,et al.  Identifying Generic Noun Phrases , 2010, ACL.

[19]  Christiane Fellbaum,et al.  MASC: the Manually Annotated Sub-Corpus of American English , 2008, LREC.

[20]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Carlota S. Smith,et al.  The Parameter of Aspect , 1991 .

[23]  Bowen Zhou,et al.  Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond , 2016, CoNLL.

[24]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[25]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[26]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[27]  Alexis Palmer,et al.  Automatic prediction of aspectual class of verbs in context , 2014, ACL.

[28]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[29]  Jason Baldridge,et al.  A Sequencing Model for Situation Entity Classification , 2007, ACL.

[30]  Vivi Nastase,et al.  Classifying Semantic Clause Types: Modeling Context and Genre Characteristics with Recurrent Neural Networks and Attention , 2017, *SEM.

[31]  Peng Zhou,et al.  Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme , 2017, ACL.

[32]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[33]  Henk J. Verkuyl,et al.  On the Compositional Nature of the Aspects , 1972 .

[34]  Hai Zhao,et al.  Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Recurrent Neural Network , 2015, ArXiv.

[35]  Carlota S. Smith,et al.  Modes of Discourse: The Local Structure of Texts , 2009 .