Semantic Role Labeling as Syntactic Dependency Parsing

We reduce the task of (span-based) PropBank-style semantic role labeling (SRL) to syntactic dependency parsing. Our approach is motivated by our empirical analysis that shows three common syntactic patterns account for over 98% of the SRL annotations for both English and Chinese data. Based on this observation, we present a conversion scheme that packs SRL annotations into dependency tree representations through joint labels that permit highly accurate recovery back to the original format. This representation allows us to train statistical dependency parsers to tackle SRL and achieve competitive performance with the current state of the art. Our findings show the promise of syntactic dependency trees in encoding semantic role relations within their syntactic domain of locality, and point to potential further integration of syntactic methods into semantic role labeling in the future.

[1]  Baobao Chang,et al.  Syntax Aware LSTM model for Semantic Role Labeling , 2017, SPNLP@EMNLP.

[2]  Ivan Titov,et al.  A Latent Variable Model of Synchronous Parsing for Syntactic and Semantic Dependencies , 2008, CoNLL.

[3]  Mark C. Baker,et al.  Incorporation: A Theory of Grammatical Function Changing , 1988 .

[4]  Raymond J. Mooney,et al.  A Statistical Semantic Parser that Integrates Syntax and Semantics , 2005, CoNLL.

[5]  Hai Zhao,et al.  Dependency or Span, End-to-End Uniform Semantic Role Labeling , 2019, AAAI.

[6]  Daniel Gildea,et al.  The Necessity of Parsing for Predicate Argument Recognition , 2002, ACL.

[7]  Anders Søgaard,et al.  Deep multi-task learning with low level tasks supervised at lower layers , 2016, ACL.

[8]  Ari Rappoport,et al.  Multitask Parsing Across Semantic Representations , 2018, ACL.

[9]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[10]  Xavier Carreras,et al.  Joint Arc-factored Parsing of Syntactic and Semantic Dependencies , 2013, Transactions of the Association for Computational Linguistics.

[11]  Nathan Schneider,et al.  A Structured Syntax-Semantics Interface for English-AMR Alignment , 2018, NAACL-HLT.

[12]  Y.-H. Audrey Li,et al.  The Syntax of Chinese , 2009 .

[13]  Roser Morante,et al.  Dependency Parsing and Semantic Role Labeling as a Single Task , 2009, RANLP.

[14]  Mark Steedman,et al.  Universal Semantic Parsing , 2017, EMNLP.

[15]  Timothy Dozat,et al.  Stanford’s Graph-based Neural Dependency Parser at the CoNLL 2017 Shared Task , 2017, CoNLL.

[16]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[17]  Daniel Gildea,et al.  Semantic Roles for String to Tree Machine Translation , 2013, ACL.

[18]  Daniel Gildea,et al.  Automatic Labeling of Semantic Roles , 2000, ACL.

[19]  Seth Kulick,et al.  Issues in Synchronizing the English Treebank and PropBank , 2006 .

[20]  Wolfgang Menzel,et al.  Joint Labeling of Syntactic Function and Semantic Role Using Probabilistic Finite State Automata , 2018, IntelliSys.

[21]  Karin Kipper Schuler,et al.  Argument Realization , 2006, Comput. Linguistics.

[22]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[23]  Yue Zhang,et al.  Dependency Tree Representations of Predicate-Argument Structures , 2016, AAAI.

[24]  Jeffrey Gruber.  Studies in lexical relations , 1965 .

[25]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[26]  R'emi Louf,et al.  HuggingFace's Transformers: State-of-the-art Natural Language Processing , 2019, ArXiv.

[27]  Lluís Màrquez i Villodre,et al.  A Joint Model for Parsing Syntactic and Semantic Dependencies , 2008, CoNLL.

[28]  Martha Palmer,et al.  Retrieving Correct Semantic Boundaries in Dependency Structure , 2010, Linguistic Annotation Workshop.

[29]  Anoop Sarkar,et al.  Exploration of the LTAG-Spinal Formalism and Treebank for Semantic Role Labeling , 2009 .

[30]  Benjamin Van Durme,et al.  The Semantic Proto-Role Linking Model , 2017, EACL.

[31]  Luo Si,et al.  Syntax-Aware Neural Semantic Role Labeling , 2019, AAAI.

[32]  Rodney D. Nielsen,et al.  Towards comprehensive syntactic and semantic annotations of the clinical narrative , 2013, J. Am. Medical Informatics Assoc..

[33]  Andrew McCallum,et al.  Linguistically-Informed Self-Attention for Semantic Role Labeling , 2018, EMNLP.

[34]  Hai Zhao,et al.  Multilingual Dependency Learning: Exploiting Rich Features for Tagging Syntactic and Semantic Dependencies , 2009, CoNLL Shared Task.

[35]  Richard Johansson,et al.  The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies , 2008, CoNLL.

[36]  Christopher D. Manning,et al.  A Global Joint Model for Semantic Role Labeling , 2008, CL.

[37]  Martin Kay,et al.  Syntactic Process , 1979, ACL.

[38]  Seth Kulick,et al.  Fully Parsing the Penn Treebank , 2006, NAACL.

[39]  Timothy Dozat,et al.  Simpler but More Accurate Semantic Dependency Parsing , 2018, ACL.

[40]  Mirella Lapata,et al.  Using Semantic Roles to Improve Question Answering , 2007, EMNLP.

[41]  Hiroyuki Shindo,et al.  A Span Selection Model for Semantic Role Labeling , 2018, EMNLP.

[42]  Natalie Schluter.  Maximising Spanning Subtree Scores for Parsing Tree Approximations of Semantic Dependency Digraphs , 2015, IWPT.

[43]  Mirella Lapata,et al.  Syntax-aware Semantic Role Labeling without Parsing , 2019, TACL.

[44]  Ding Liu,et al.  Semantic Role Features for Machine Translation , 2010, COLING.

[45]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[46]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[47]  Dan Klein,et al.  Constituency Parsing with a Self-Attentive Encoder , 2018, ACL.

[48]  Weiwei Sun,et al.  Prediction of Maximal Projection for Semantic Role Labeling , 2008, COLING.

[49]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[50]  Martha Palmer,et al.  From TreeBank to PropBank , 2002, LREC.

[51]  Lillian Lee,et al.  Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set , 2017, EMNLP.

[52]  Luke S. Zettlemoyer,et al.  Deep Semantic Role Labeling: What Works and What’s Next , 2017, ACL.

[53]  Ivan Titov,et al.  Multilingual Joint Parsing of Syntactic and Semantic Dependencies with a Latent Variable Model , 2013, CL.

[54]  Nianwen Xue,et al.  Labeling Chinese Predicates with Semantic Roles , 2008, CL.

[55]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[56]  Dan Roth,et al.  Question Answering as Global Reasoning Over Semantic Abstractions , 2018, AAAI.

[57]  Richard Johansson.  Statistical Bistratal Dependency Parsing , 2009, EMNLP.

[58]  Gabriele Musillo,et al.  Semantic Parsing for High-Precision Semantic Role Labelling , 2008, CoNLL.

[59]  Christopher D. Manning,et al.  Enhanced English Universal Dependencies: An Improved Representation for Natural Language Understanding Tasks , 2016, LREC.

[60]  Daniel Gildea,et al.  Identifying Semantic Roles Using Combinatory Categorial Grammar , 2003, EMNLP.

[61]  Ari Rappoport,et al.  Content Differences in Syntactic and Semantic Representation , 2019, NAACL-HLT.

[62]  Kuzman Ganchev,et al.  Semantic Role Labeling with Neural Network Factors , 2015, EMNLP.

[63]  Weiwei Sun,et al.  Peking: Profiling Syntactic Tree Parsing Techniques for Semantic Graph Parsing , 2014, SemEval@COLING.

[64]  Haizhou Li,et al.  Modeling the Translation of Predicate-Argument Structure for SMT , 2012, ACL.

[65]  Gerold Schneider,et al.  Hybrid Long-Distance Functional Dependency Parsing , 2009 .

[66]  Dirk Hovy,et al.  Copenhagen-Malmö: Tree Approximations of Semantic Parsing Problems , 2014, SemEval@COLING.

[67]  Christopher R. Johnson,et al.  Background to Framenet , 2003 .

[68]  M. A. R T A P A L,et al.  The Penn Chinese TreeBank: Phrase structure annotation of a large corpus , 2005, Natural Language Engineering.

[69]  Hai Zhao,et al.  A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware? , 2018, COLING.

[70]  Hwee Tou Ng,et al.  Joint Syntactic and Semantic Parsing of Chinese , 2010, ACL.

[71]  Luke S. Zettlemoyer,et al.  Syntactic Scaffolds for Semantic Structures , 2018, EMNLP.

[72]  Philipp Koehn,et al.  Synthesis Lectures on Human Language Technologies , 2016 .

[73]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[74]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[75]  Claire Bonial,et al.  English PropBank Annotation Guidelines , 2012 .

[76]  Nianwen Xue,et al.  Annotating the Propositions in the Penn Chinese Treebank , 2003, SIGHAN.

[77]  Joakim Nivre,et al.  Dependency Parsing , 2009, Lang. Linguistics Compass.

[78]  Mirella Lapata,et al.  Neural Semantic Role Labeling with Dependency Path Embeddings , 2016, ACL.

[79]  Ivan Titov,et al.  Semantic Role Labeling , 2010, HLT-NAACL.

[80]  Xavier Carreras,et al.  Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling , 2005, CoNLL.

[81]  Christopher D. Manning,et al.  Stanford typed dependencies manual , 2010 .

[82]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[83]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[84]  Charles N. Li,et al.  Mandarin Chinese: A Functional Reference Grammar , 1989 .

[85]  Luke S. Zettlemoyer,et al.  Joint A* CCG Parsing and Semantic Role Labelling , 2015, EMNLP.

[86]  Yue Zhang,et al.  Design Challenges and Misconceptions in Neural Sequence Labeling , 2018, COLING.

[87]  Gabriele Musillo,et al.  Accurate Parsing of the Proposition Bank , 2006, HLT-NAACL.

[88]  Samuel R. Bowman,et al.  A Gold Standard Dependency Corpus for English , 2014, LREC.

[89]  Hai Zhao,et al.  Syntax for Semantic Role Labeling, To Be, Or Not To Be , 2018, ACL.

[90]  C. Fillmore.  FRAME SEMANTICS AND THE NATURE OF LANGUAGE * , 1976 .

[91]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[92]  Frank Keller,et al.  Incremental Semantic Role Labeling with Tree Adjoining Grammar , 2014, EMNLP.

[93]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[94]  Yudong Liu.  Semantic role labeling using lexicalized tree adjoining grammars , 2009 .

[95]  Yue Zhang,et al.  Exploiting Mutual Benefits between Syntax and Semantic Roles using Neural Network , 2016, EMNLP.

[96]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[97]  Omer Levy,et al.  Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling , 2018, ACL.

[98]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[99]  Ralf Zimmer,et al.  RelEx - Relation extraction using dependency parse trees , 2007, Bioinform..

[100]  Mark C. Baker,et al.  Thematic Roles and Syntactic Structure , 1997 .

[101]  Naomie Salim,et al.  A framework for multi-document abstractive summarization based on semantic role labelling , 2015, Appl. Soft Comput..

[102]  Dan Roth,et al.  The Importance of Syntactic Parsing and Inference in Semantic Role Labeling , 2008, CL.

[103]  Noah A. Smith,et al.  Deep Multitask Learning for Semantic Dependency Parsing , 2017, ACL.

[104]  Giorgio Satta,et al.  Efficient Parsing for Bilexical Context-Free Grammars and Head Automaton Grammars , 1999, ACL.

[105]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.

[106]  Hai Zhao,et al.  A Unified Syntax-aware Framework for Semantic Role Labeling , 2018, EMNLP.

[107]  Noah A. Smith,et al.  Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs , 2016, CoNLL.

[108]  Oren Etzioni,et al.  Semantic Role Labeling for Open Information Extraction , 2010, HLT-NAACL 2010.

[109]  Daniel Jurafsky,et al.  Semantic Role Labeling Using Different Syntactic Views , 2005, ACL.

[110]  Dan Klein,et al.  Parsing with Traces: An O(n4) Algorithm and a Structural Representation , 2017, TACL.

[111]  Aravind K. Joshi,et al.  Tree Adjunct Grammars , 1975, J. Comput. Syst. Sci..

[112]  Mark C. Baker,et al.  On the Structural Positions of Themes and Goals , 1996 .

[113]  Christopher D. Manning,et al.  Generating Typed Dependency Parses from Phrase Structure Parses , 2006, LREC.

[114]  Gabriele Musillo,et al.  Accurate Function Parsing , 2005, HLT/EMNLP.

[115]  Yuchen Zhang,et al.  CoNLL-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes , 2012, EMNLP-CoNLL Shared Task.

[116]  Richard Johansson,et al.  The Effect of Syntactic Representation on Semantic Role Labeling , 2008, COLING.

[117]  Olga Babko-Malaya,et al.  PropBank Annotation Guidelines , 2010 .

[118]  Martha Palmer,et al.  The Integration of Syntactic Parsing and Semantic Role Labeling , 2005, CoNLL.

[119]  Mitchell P. Marcus,et al.  OntoNotes: The 90% Solution , 2006, NAACL.

[120]  Kuzman Ganchev,et al.  Efficient Inference and Structured Learning for Semantic Role Labeling , 2015, TACL.

[121]  Daniel Kondratyuk,et al.  75 Languages, 1 Model: Parsing Universal Dependencies Universally , 2019, EMNLP.

[122]  Ray Jackendoff,et al.  Semantic Interpretation in Generative Grammar , 1972 .

[123]  Lillian Lee,et al.  Valency-Augmented Dependency Parsing , 2018, EMNLP.

[124]  Xavier Carreras,et al.  Semantic Role Labeling: An Introduction to the Special Issue , 2008, Computational Linguistics.

[125]  Dan Roth,et al.  The Necessity of Syntactic Parsing for Semantic Role Labeling , 2005, IJCAI.

[126]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[127]  Jungo Kasai,et al.  Syntax-aware Neural Semantic Role Labeling with Supertags , 2019, NAACL.

[128]  Dan Klein,et al.  Multilingual Constituency Parsing with Self-Attention and Pre-Training , 2018, ACL.

[129]  Kadri Hacioglu,et al.  Semantic Role Labeling Using Dependency Trees , 2004, COLING.

[130]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[131]  Yidong Chen,et al.  Deep Semantic Role Labeling with Self-Attention , 2017, AAAI.

[132]  Min Zhang,et al.  A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling , 2019, EMNLP.

[133]  Johan Eklund,et al.  Mixing and Blending Syntactic and Semantic Dependencies , 2008, CoNLL.

[134]  Daniel Jurafsky,et al.  Discriminative Reordering with Chinese Grammatical Relations Features , 2009, SSST@HLT-NAACL.

[135]  Reid Swanson,et al.  A Comparison of Alternative Parse Tree Paths for Labeling Semantic Roles , 2006, ACL.

[136]  Wei Xu,et al.  End-to-end learning of semantic role labeling using recurrent neural networks , 2015, ACL.

[137]  John S. Bowers,et al.  Arguments as Relations , 2010 .

[138]  Yufei Wang,et al.  How to best use Syntax in Semantic Role Labelling , 2019, ACL.

[139]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[140]  Vivek Srikumar,et al.  Structured Tuning for Semantic Role Labeling , 2020, ACL.

[141]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.