Semantic Role Labeling with Associated Memory Network

Semantic role labeling (SRL) is a task to recognize all the predicate-argument pairs of a sentence, which has been in a performance improvement bottleneck after a series of latest works were presented. This paper proposes a novel syntax-agnostic SRL model enhanced by the proposed associated memory network (AMN), which makes use of inter-sentence attention of label-known associated sentences as a kind of memory to further enhance dependency-based SRL. In detail, we use sentences and their labels from train dataset as an associated memory cue to help label the target sentence. Furthermore, we compare several associated sentences selecting strategies and label merging methods in AMN to find and utilize the label of associated sentences while attending them. By leveraging the attentive memory from known training data, Our full model reaches state-of-the-art on CoNLL-2009 benchmark datasets for syntax-agnostic setting, showing a new effective research line of SRL enhancement other than exploiting external resources such as well pre-trained language models.

[1]  Hong Yu,et al.  Learning for Biomedical Information Extraction: Methodological Review of Recent Advances , 2016, ArXiv.

[2]  Daniel Jurafsky,et al.  Semantic Role Labeling Using Different Syntactic Views , 2005, ACL.

[3]  Hai Zhao,et al.  Integrative Semantic Dependency Parsing via Efficient Large-scale Feature Selection , 2013, J. Artif. Intell. Res..

[4]  Michael Roth,et al.  Composition of Word Representations Improves Semantic Role Labelling , 2014, EMNLP.

[5]  Sanjeev Arora,et al.  A Simple but Tough-to-Beat Baseline for Sentence Embeddings , 2017, ICLR.

[6]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[7]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[8]  Andrew McCallum,et al.  Linguistically-Informed Self-Attention for Semantic Role Labeling , 2018, EMNLP.

[9]  Gerhard Weikum,et al.  Automated Template Generation for Question Answering over Knowledge Graphs , 2017, WWW.

[10]  Luke S. Zettlemoyer,et al.  Deep Semantic Role Labeling: What Works and What’s Next , 2017, ACL.

[11]  Jindrich Libovický,et al.  Attention Strategies for Multi-Source Sequence-to-Sequence Learning , 2017, ACL.

[12]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[13]  Omer Levy,et al.  Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling , 2018, ACL.

[14]  Hai Zhao,et al.  Semantic Dependency Parsing of NomBank and PropBank: An Efficient Integrated Approach via a Large-scale Feature Selection , 2009, EMNLP.

[15]  Claire Bonial,et al.  English PropBank Annotation Guidelines , 2012 .

[16]  Yijia Liu,et al.  Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation , 2018, CoNLL.

[17]  Hai Zhao,et al.  Syntax for Semantic Role Labeling, To Be, Or Not To Be , 2018, ACL.

[18]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[19]  Hai Zhao,et al.  Multilingual Dependency Learning: A Huge Feature Engineering Method to Semantic Dependency Parsing , 2009, CoNLL Shared Task.

[20]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[21]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Hai Zhao,et al.  Dependency or Span, End-to-End Uniform Semantic Role Labeling , 2019, AAAI.

[24]  Matt J. Kusner,et al.  From Word Embeddings To Document Distances , 2015, ICML.

[25]  Hai Zhao,et al.  A Unified Syntax-aware Framework for Semantic Role Labeling , 2018, EMNLP.

[26]  Diego Marcheggiani,et al.  A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling , 2017, CoNLL.

[27]  Kuzman Ganchev,et al.  Semantic Role Labeling with Neural Network Factors , 2015, EMNLP.

[28]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[29]  Hai Zhao,et al.  A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware? , 2018, COLING.

[30]  Christopher D. Manning,et al.  A Global Joint Model for Semantic Role Labeling , 2008, CL.

[31]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[32]  Hai Zhao,et al.  Multilingual Dependency Learning: Exploiting Rich Features for Tagging Syntactic and Semantic Dependencies , 2009, CoNLL Shared Task.

[33]  Yidong Chen,et al.  Deep Semantic Role Labeling with Self-Attention , 2017, AAAI.

[34]  Murhaf Fares,et al.  Word vectors, reuse, and replicability: Towards a community repository of large-text resources , 2017, NODALIDA.

[35]  Mirella Lapata,et al.  Neural Semantic Role Labeling with Dependency Path Embeddings , 2016, ACL.

[36]  Jason Weston,et al.  Key-Value Memory Networks for Directly Reading Documents , 2016, EMNLP.

[37]  Wei Xu,et al.  End-to-end learning of semantic role labeling using recurrent neural networks , 2015, ACL.