End-to-end Semantic Role Labeling with Neural Transition-based Model

End-to-end semantic role labeling (SRL) has been received increasing interest. It performs the two subtasks of SRL: predicate identification and argument role labeling, jointly. Recent work is mostly focused on graph-based neural models, while the transition-based framework with neural networks which has been widely used in a number of closely-related tasks, has not been studied for the joint task yet. In this paper, we present the first work of transition-based neural models for end-to-end SRL. Our transition model incrementally discovers all sentential predicates as well as their arguments by a set of transition actions. The actions of the two subtasks are executed mutually for full interactions. Besides, we suggest high-order compositions to extract non-local features, which can enhance the proposed transition model further. Experimental results on CoNLL09 and Universal Proposition Bank show that our final model can produce state-of-the-art performance, and meanwhile keeps highly efficient in decoding. We also conduct detailed experimental analysis for a deep understanding of our proposed model.

[1]  Pierre Nugues,et al.  A High-Performance Syntactic and Semantic Dependency Parser , 2010, COLING.

[2]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[3]  Deng Cai,et al.  Core Semantic First: A Top-down Approach for AMR Parsing , 2019, EMNLP.

[4]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[5]  Mirella Lapata,et al.  Using Semantic Roles to Improve Question Answering , 2007, EMNLP.

[6]  Yue Zhang,et al.  Joint Word Segmentation, POS-Tagging and Syntactic Chunking , 2016, AAAI.

[7]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[8]  Mirella Lapata,et al.  Neural Semantic Role Labeling with Dependency Path Embeddings , 2016, ACL.

[9]  Shi Feng,et al.  Knowledge-Based Semantic Embedding for Machine Translation , 2016, ACL.

[10]  Hai Zhao,et al.  Dependency or Span, End-to-End Uniform Semantic Role Labeling , 2019, AAAI.

[11]  Hai Zhao,et al.  Multilingual Dependency Learning: Exploiting Rich Features for Tagging Syntactic and Semantic Dependencies , 2009, CoNLL Shared Task.

[12]  Alessandro Moschitti,et al.  High-Order Low-Rank Tensors for Semantic Role Labeling , 2015, HLT-NAACL.

[13]  Stephen Clark,et al.  A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing , 2008, EMNLP.

[14]  Kewei Tu,et al.  Bidirectional Transition-Based Dependency Parsing , 2019, AAAI.

[15]  Daniel Jurafsky,et al.  Semantic Role Labeling Using Different Syntactic Views , 2005, ACL.

[16]  Yunyao Li,et al.  POLYGLOT: Multilingual Semantic Role Labeling with Unified Labels , 2016, ACL.

[17]  Oren Etzioni,et al.  An analysis of open information extraction based on semantic role labeling , 2011, K-CAP '11.

[18]  Daniel Gildea,et al.  Automatic Labeling of Semantic Roles , 2000, ACL.

[19]  Anders Søgaard,et al.  Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies , 2019, ACL.

[20]  Stephen Clark,et al.  A Fast Decoder for Joint Word Segmentation and POS-Tagging Using a Single Discriminative Model , 2010, EMNLP.

[21]  Diego Marcheggiani,et al.  A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling , 2017, CoNLL.

[22]  Hai Zhao,et al.  A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware? , 2018, COLING.

[23]  Haizhou Li,et al.  Modeling the Translation of Predicate-Argument Structure for SMT , 2012, ACL.

[24]  Yoav Goldberg,et al.  An Efficient Algorithm for Easy-First Non-Directional Dependency Parsing , 2010, NAACL.

[25]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[26]  Yue Zhang,et al.  Transition-Based Neural Word Segmentation , 2016, ACL.

[27]  Dan Roth,et al.  The Importance of Syntactic Parsing and Inference in Semantic Role Labeling , 2008, CL.

[28]  Yunyao Li,et al.  Generating High Quality Proposition Banks for Multilingual Semantic Role Labeling , 2015, ACL.

[29]  Roberto Basili,et al.  Textual Inference and Meaning Representation in Human Robot Interaction , 2013, JSSP.

[30]  Dan Roth,et al.  Semantic Role Labeling Via Generalized Inference Over Classifiers , 2004, CoNLL.

[31]  Luo Si,et al.  Syntax-Aware Neural Semantic Role Labeling , 2019, AAAI.

[32]  Makoto Miwa,et al.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures , 2016, ACL.

[33]  Donghong Ji,et al.  Extracting Entities and Events as a Single Task Using a Transition-Based Neural Model , 2019, IJCAI.

[34]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[35]  Donghong Ji,et al.  Cross-Lingual Semantic Role Labeling With Model Transfer , 2020, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[36]  Hongxia Jin,et al.  A Neural Transition-based Model for Nested Mention Recognition , 2018, EMNLP.

[37]  Donghong Ji,et al.  Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus , 2020, ACL.

[38]  Kuzman Ganchev,et al.  Semantic Role Labeling with Neural Network Factors , 2015, EMNLP.

[39]  Shay B. Cohen,et al.  Semantic Role Labeling with Iterative Structure Refinement , 2019, EMNLP.

[40]  Andrew McCallum,et al.  Linguistically-Informed Self-Attention for Semantic Role Labeling , 2018, EMNLP.

[41]  Richard Johansson,et al.  The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages , 2009, CoNLL Shared Task.

[42]  Martha Palmer,et al.  Transition-based Semantic Role Labeling Using Predicate Argument Clustering , 2011, RELMS@ACL.

[43]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[44]  Yue Zhang,et al.  A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing , 2015, ACL.

[45]  Luo Si,et al.  Syntax-Enhanced Self-Attention-Based Semantic Role Labeling , 2019, EMNLP/IJCNLP.

[46]  Omer Levy,et al.  Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling , 2018, ACL.

[47]  Christian Scheible,et al.  An Evaluation of Predicate Argument Clustering using Pseudo-Disambiguation , 2010, LREC.

[48]  Donghong Ji,et al.  High-order Refining for End-to-end Chinese Semantic Role Labeling , 2020, AACL/IJCNLP.

[49]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[50]  Yue Zhang,et al.  Transition-Based Neural Word Segmentation Using Word-Level Features , 2018, J. Artif. Intell. Res..

[51]  Andrew Chou,et al.  Semantic Parsing on Freebase from Question-Answer Pairs , 2013, EMNLP.