High-order Semantic Role Labeling

Semantic role labeling is primarily used to identify predicates, arguments, and their semantic relationships. Due to the limitations of modeling methods and the conditions of pre-identified predicates, previous work has focused on the relationships between predicates and arguments and the correlations between arguments at most, while the correlations between predicates have been neglected for a long time. High-order features and structure learning were very common in modeling such correlations before the neural network era. In this paper, we introduce a high-order graph structure for the neural semantic role labeling model, which enables the model to explicitly consider not only the isolated predicate-argument pairs but also the interaction between the predicate-argument pairs. Experimental results on 7 languages of the CoNLL-2009 benchmark show that the high-order structural learning techniques are beneficial to the strong performing SRL models and further boost our baseline to achieve new state-of-the-art results.

[1]  Noah A. Smith,et al.  Polyglot Semantic Role Labeling , 2018, ACL.

[2]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[3]  Hai Zhao,et al.  Hierarchical Contextualized Representation for Named Entity Recognition , 2019, AAAI.

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  Luke S. Zettlemoyer,et al.  Higher-Order Coreference Resolution with Coarse-to-Fine Inference , 2018, NAACL.

[6]  Mirella Lapata,et al.  Neural Semantic Role Labeling with Dependency Path Embeddings , 2016, ACL.

[7]  Hai Zhao,et al.  Semantics-aware BERT for Language Understanding , 2020, AAAI.

[8]  Min Zhang,et al.  Efficient Second-Order TreeCRF for Neural Dependency Parsing , 2020, ACL.

[9]  Wei Xu,et al.  End-to-end learning of semantic role labeling using recurrent neural networks , 2015, ACL.

[10]  Philipp Koehn,et al.  Statistical Significance Tests for Machine Translation Evaluation , 2004, EMNLP.

[11]  Kehai Chen,et al.  Lattice-Based Transformer Encoder for Neural Machine Translation , 2019, ACL.

[12]  Junru Zhou,et al.  Parsing All: Syntax and Semantics, Dependencies and Spans , 2020, EMNLP.

[13]  Hao Zhang,et al.  Generalized Higher-Order Dependency Parsing with Cube Pruning , 2012, EMNLP.

[14]  Hai Zhao,et al.  Semantic Dependency Parsing of NomBank and PropBank: An Efficient Integrated Approach via a Large-scale Feature Selection , 2009, EMNLP.

[15]  Junru Zhou,et al.  Head-Driven Phrase Structure Grammar Parsing on Penn Treebank , 2019, ACL.

[16]  Hai Zhao,et al.  Multilingual Dependency Learning: Exploiting Rich Features for Tagging Syntactic and Semantic Dependencies , 2009, CoNLL Shared Task.

[17]  André F. T. Martins,et al.  Priberam: A Turbo Semantic Parser with Second Order Features , 2014, *SEMEVAL.

[18]  Mark Dredze,et al.  Approximation-Aware Dependency Parsing by Belief Propagation , 2015, TACL.

[19]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[20]  Hai Zhao,et al.  Parsing Syntactic and Semantic Dependencies with Two Single-Stage Maximum Entropy Models , 2008, CoNLL.

[21]  Michael C. Mozer,et al.  A Focused Backpropagation Algorithm for Temporal Pattern Recognition , 1989, Complex Syst..

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Mirella Lapata,et al.  Semi-Supervised Semantic Role Labeling with Cross-View Training , 2019, EMNLP.

[24]  Hiroyuki Shindo,et al.  Neural Modeling of Multi-Predicate Interactions for Japanese Predicate Argument Structure Analysis , 2017, ACL.

[25]  Hiroyuki Shindo,et al.  Joint Case Argument Identification for Japanese Predicate Argument Structure Analysis , 2015, ACL.

[26]  Hai Zhao,et al.  Global Greedy Dependency Parsing , 2020, AAAI.

[27]  Jungo Kasai,et al.  Syntax-aware Neural Semantic Role Labeling with Supertags , 2019, NAACL.

[28]  Hai Zhao,et al.  Fourth-Order Dependency Parsing , 2012, COLING.

[29]  Noah A. Smith,et al.  Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs , 2016, CoNLL.

[30]  Zhuosheng Zhang,et al.  Effective Subword Segmentation for Text Comprehension , 2018, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[31]  Maria Leonor Pacheco,et al.  of the Association for Computational Linguistics: , 2001 .

[32]  Xinchi Chen,et al.  Capturing Argument Interaction in Semantic Role Labeling with Capsule Networks , 2019, EMNLP.

[33]  Kewei Tu,et al.  Second-Order Semantic Dependency Parsing with End-to-End Neural Networks , 2019, ACL.

[34]  Yuji Matsumoto,et al.  Jointly Extracting Japanese Predicate-Argument Relation with Markov Logic , 2011, IJCNLP.

[35]  Shay B. Cohen,et al.  Semantic Role Labeling with Iterative Structure Refinement , 2019, EMNLP.

[36]  Hai Zhao,et al.  A Unified Syntax-aware Framework for Semantic Role Labeling , 2018, EMNLP.

[37]  Hai Zhao,et al.  Syntax for Semantic Role Labeling, To Be, Or Not To Be , 2018, ACL.

[38]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[40]  Alexey Sorokin,et al.  Tuning Multilingual Transformers for Language-Specific Named Entity Recognition , 2019, BSNLP@ACL.

[41]  Noah A. Smith,et al.  Dual Decomposition with Many Overlapping Components , 2011, EMNLP.

[42]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[43]  David A. Smith,et al.  Dependency Parsing by Belief Propagation , 2008, EMNLP.

[44]  Eric P. Xing,et al.  Concise Integer Linear Programming Formulations for Dependency Parsing , 2009, ACL.

[45]  Prakhar Gupta,et al.  Learning Word Vectors for 157 Languages , 2018, LREC.

[46]  Omer Levy,et al.  Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling , 2018, ACL.

[47]  Yoshimasa Tsuruoka,et al.  Improving Graph-based Dependency Parsing with Decision History , 2010, COLING.

[48]  Hai Zhao,et al.  Dependency or Span, End-to-End Uniform Semantic Role Labeling , 2019, AAAI.

[49]  Yuji Matsumoto,et al.  Annotating a Japanese Text Corpus with Predicate-Argument and Coreference Relations , 2007, LAW@ACL.

[50]  Kentaro Inui,et al.  Distance-Free Modeling of Multi-Predicate Interactions in End-to-End Japanese Predicate-Argument Structure Analysis , 2018, COLING.

[51]  Luke S. Zettlemoyer,et al.  Deep Semantic Role Labeling: What Works and What’s Next , 2017, ACL.

[52]  Fernando Pereira,et al.  Online Learning of Approximate Dependency Parsing Algorithms , 2006, EACL.

[53]  Diego Marcheggiani,et al.  A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling , 2017, CoNLL.

[54]  Daisuke Kawahara,et al.  Neural Network-Based Model for Japanese Predicate Argument Structure Analysis , 2016, ACL.

[55]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[56]  Xavier Carreras,et al.  Experiments with a Higher-Order Projective Dependency Parser , 2007, EMNLP.

[57]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[58]  Pierre Nugues,et al.  Multilingual Semantic Role Labeling , 2009, CoNLL Shared Task.

[59]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[60]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[61]  Andrew McCallum,et al.  Linguistically-Informed Self-Attention for Semantic Role Labeling , 2018, EMNLP.

[62]  Daniel Gildea,et al.  Automatic Labeling of Semantic Roles , 2000, ACL.

[63]  Hai Zhao,et al.  Integrative Semantic Dependency Parsing via Efficient Large-scale Feature Selection , 2013, J. Artif. Intell. Res..

[64]  Weiwei Sun,et al.  Quasi-Second-Order Parsing for 1-Endpoint-Crossing, Pagenumber-2 Graphs , 2017, EMNLP.

[65]  Hai Zhao,et al.  Seq2seq Dependency Parsing , 2018, COLING.

[66]  Masao Utiyama,et al.  SJTU-NICT at MRP 2019: Multi-Task Learning for End-to-End Uniform Semantic Graph Parsing , 2019, CoNLL.

[67]  Jong-Hoon Oh,et al.  Intra-sentential Zero Anaphora Resolution using Subject Sharing Recognition , 2015, EMNLP.

[68]  Hai Zhao,et al.  A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware? , 2018, COLING.