Valency-Augmented Dependency Parsing

We present a complete, automated, and efficient approach for utilizing valency analysis in making dependency parsing decisions. It includes extraction of valency patterns, a probabilistic model for tagging these patterns, and a joint decoding process that explicitly considers the number and types of each token’s syntactic dependents. On 53 treebanks representing 41 languages in the Universal Dependencies data, we find that incorporating valency information yields higher precision and F1 scores on the core arguments (subjects and complements) and functional relations (e.g., auxiliaries) that we employ for valency analysis. Precision on core arguments improves from 80.87 to 85.43. We further show that our approach can be applied to an ostensibly different formalism and dataset, Tree Adjoining Grammar as extracted from the Penn Treebank; there, we outperform the previous state-of-the-art labeled attachment score by 0.7. Finally, we explore the potential of extending valency patterns beyond their traditional domain by confirming their helpfulness in improving PP attachment decisions.

[1]  Mark Steedman,et al.  Universal Semantic Parsing , 2017, EMNLP.

[2]  Mark Steedman,et al.  Using CCG categories to improve Hindi dependency parsing , 2013, ACL.

[3]  Jaime G. Carbonell,et al.  A Discriminative Graph-Based Parser for the Abstract Meaning Representation , 2014, ACL.

[4]  Richard Johansson,et al.  The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies , 2008, CoNLL.

[5]  Kuzman Ganchev,et al.  Efficient Inference and Structured Learning for Semantic Role Labeling , 2015, TACL.

[6]  Noah A. Smith,et al.  Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs , 2015, EMNLP.

[7]  Alon Lavie,et al.  A Best-First Probabilistic Shift-Reduce Parser , 2006, ACL.

[8]  Hiroyuki Shindo,et al.  Transition-Based Dependency Parsing Exploiting Supertags , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[9]  Michael Collins,et al.  Prepositional Phrase Attachment through a Backed-off Model , 1995, VLC@ACL.

[10]  Agnieszka Falenska,et al.  Stacking or Supertagging for Dependency Parsing - What's the Difference? , 2015, IWPT.

[11]  Eugene Charniak,et al.  Figures of Merit for Best-First Probabilistic Chart Parsing , 1998, Comput. Linguistics.

[12]  Rajeev Sangal,et al.  Linguistically Rich Graph Based Data Driven Parsing For Hindi , 2011, SPMRL@IWPT.

[13]  Yusuke Miyao,et al.  SemEval 2015 Task 18: Broad-Coverage Semantic Dependency Parsing , 2015, *SEMEVAL.

[14]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[15]  Dan Klein,et al.  K-Best A* Parsing , 2009, ACL.

[16]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[17]  Joseph Le Roux,et al.  Semi-supervised Dependency Parsing using Lexical Affinities , 2012, ACL.

[18]  Adwait Ratnaparkhi,et al.  A Linear Observed Time Statistical Parser Based on Maximum Entropy Models , 1997, EMNLP.

[19]  Hiyan Alshawi,et al.  Head Automata and Bilingual Tiling: Translation with Minimal Representations , 1996, ACL.

[20]  Luke S. Zettlemoyer,et al.  LSTM CCG Parsing , 2016, NAACL.

[21]  Jun'ichi Tsujii,et al.  Extremely Lexicalized Models for Accurate and Fast HPSG Parsing , 2006, EMNLP.

[22]  Joakim Nivre,et al.  Dependency Grammar and Dependency Parsing , 2005 .

[23]  Dan Klein,et al.  Parser Showdown at the Wall Street Corral: An Empirical Investigation of Error Types in Parser Output , 2012, EMNLP.

[24]  Aravind K. Joshi,et al.  A Formal Look at Dependency Grammars and Phrase-Structure Grammars, with Special Consideration of Word-Order Phenomena , 1994, ArXiv.

[25]  Jungo Kasai,et al.  End-to-End Graph-Based TAG Parsing with Neural Networks , 2018, NAACL.

[26]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[27]  Wen Wang,et al.  A Statistical Constraint Dependency Grammar (CDG) Parser , 2004 .

[28]  Joakim Nivre,et al.  When word order and part-of-speech tags are not enough - Swedish dependency parsing with rich linguistic features , 2007 .

[29]  Alexis Nasr,et al.  SuperTagging and Full Parsing , 2004, TAG+.

[30]  Alexis Nasr,et al.  MICA: A Probabilistic Dependency Parser Based on Tree Insertion Grammars (Application Note) , 2009, HLT-NAACL.

[31]  Michael Collins,et al.  Three Generative, Lexicalised Models for Statistical Parsing , 1997, ACL.

[32]  Gerhard Helbig,et al.  W? orterbuch zur Valenz und Distribution deutscher Verben , 1975 .

[33]  Thierry Poibeau,et al.  A System for Multilingual Dependency Parsing based on Bidirectional LSTM Feature Representations , 2017, CoNLL Shared Task.

[34]  Wolfgang Menzel,et al.  Guiding a Constraint Dependency Parser with Supertags , 2006, ACL.

[35]  James Cross,et al.  Incremental Parsing with Minimal Features Using Bi-Directional LSTM , 2016, ACL.

[36]  Jakub Waszczuk,et al.  Multiword Expression-Aware A$*$ TAG Parsing Revisited , 2017, TAG.

[37]  Xiang Yu,et al.  IMS at the CoNLL 2017 UD Shared Task: CRFs and Perceptrons Meet Neural Networks , 2017, CoNLL.

[38]  Lucien Tesnière.  Éléments de syntaxe structurale , 1959 .

[39]  James R. Curran,et al.  Investigating GIS and Smoothing for Maximum Entropy Taggers , 2003, EACL.

[40]  James R. Curran,et al.  Multi-Tagging for Lexicalized-Grammar Parsing , 2006, ACL.

[41]  Dan Klein,et al.  A Minimal Span-Based Neural Constituency Parser , 2017, ACL.

[42]  Damon Tutunjian,et al.  Do We Need a Distinction between Arguments and Adjuncts? Evidence from Psycholinguistic Studies of Comprehension , 2008, Lang. Linguistics Compass.

[43]  Joakim Nivre,et al.  Universal Dependency Evaluation , 2017, UDW@NoDaLiDa.

[44]  Jun'ichi Tsujii,et al.  Dependency Parsing and Domain Adaptation with LR Models and Parser Ensembles , 2007, EMNLP.

[45]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[46]  Jungo Kasai,et al.  TAG Parsing with Neural Networks and Vector Representations of Supertags , 2017, EMNLP.

[47]  Kai Zhao,et al.  Optimal Incremental Parsing via Best-First Dynamic Programming , 2013, EMNLP.

[48]  Eric Brill,et al.  A Rule-Based Approach to Prepositional Phrase Attachment Disambiguation , 1994, COLING.

[49]  David Heath,et al.  A valency dictionary of English: a corpus-based analysis of the complementation patterns of English verbs, nouns and adjectives , 2004 .

[50]  Alexis Nasr,et al.  Integrating Selectional Constraints and Subcategorization Frames in a Dependency Parser , 2016, Computational Linguistics.

[51]  Alexis Nasr,et al.  Enforcing Subcategorization Constraints in a Parser Using Sub-parses Recombining , 2013, HLT-NAACL.

[52]  Noah A. Smith,et al.  Deep Multitask Learning for Semantic Dependency Parsing , 2017, ACL.

[53]  Yao Cheng,et al.  Combining Global Models for Parsing Universal Dependencies , 2017, CoNLL.

[54]  Giorgio Satta,et al.  Efficient Parsing for Bilexical Context-Free Grammars and Head Automaton Grammars , 1999, ACL.

[55]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[56]  Mats Rooth,et al.  Structural Ambiguity and Lexical Relations , 1991, ACL.

[57]  Stephan Oepen,et al.  Broad-Coverage Semantic Dependency Parsing , 2014 .

[58]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.

[59]  Wolfgang Menzel,et al.  Hybrid Parsing: Using Probabilistic Models as Predictors for a Symbolic Parser , 2006, ACL.

[60]  Mirella Lapata,et al.  Dependency Parsing as Head Selection , 2016, EACL.

[61]  Johannes Heinecke,et al.  Eliminative Parsing with Graded Constraints , 1998, COLING-ACL.

[62]  Adam Lopez,et al.  Efficient CCG Parsing: A* versus Adaptive Supertagging , 2011, ACL.

[63]  Vojtech Kovár,et al.  Enhancing Czech Parsing with Verb Valency Frames , 2013, CICLing.

[64]  Ashish Vaswani,et al.  Efficient Structured Inference for Transition-Based Parsing with Neural Networks and Error States , 2016, Transactions of the Association for Computational Linguistics.

[65]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[66]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.

[67]  Jay Earley,et al.  An efficient context-free parsing algorithm , 1970, Commun. ACM.

[68]  Vijay K. Shanker,et al.  Towards efficient statistical parsing using lexicalized grammatical information , 2002 .

[69]  Hiroshi Maruyama,et al.  Structural Disambiguation With Constraint Propagation , 1990, ACL.

[70]  Jakub Waszczuk,et al.  Promoting multiword expressions in A* TAG parsing , 2016, COLING.

[71]  Vilmos Ágel,et al.  Dependency Grammar and Valency Theory , 2009 .

[72]  Nizar Habash,et al.  CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies , 2017, CoNLL.

[73]  Eliyahu Kiperwasser,et al.  Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations , 2016, TACL.

[74]  Srinivas Bangalore,et al.  Supertagging: An Approach to Almost Parsing , 1999, CL.

[75]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[76]  Mark Steedman,et al.  A* CCG Parsing with a Supertag-factored Model , 2014, EMNLP.

[77]  Mark Steedman,et al.  Transforming Dependency Structures to Logical Forms for Semantic Parsing , 2016, TACL.

[78]  Yuji Matsumoto,et al.  A* CCG Parsing with a Supertag and Dependency Factored Model , 2017, ACL.

[79]  James R. Curran,et al.  Identifying Cascading Errors using Constraints in Dependency Parsing , 2015, ACL.

[80]  Lillian Lee,et al.  Fast(er) Exact Decoding and Global Training for Transition-Based Dependency Parsing via a Minimal Feature Set , 2017, EMNLP.

[81]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[82]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[83]  Dan Roth,et al.  The Importance of Syntactic Parsing and Inference in Semantic Role Labeling , 2008, CL.

[84]  Dan Klein,et al.  A* Parsing: Fast Exact Viterbi Parse Selection , 2003, NAACL.

[85]  Yuji Matsumoto,et al.  Improving Dependency Parsers with Supertags , 2014, EACL.

[86]  Jungo Kasai,et al.  Linguistically Rich Vector Representations of Supertags for TAG Parsing , 2017, TAG.

[87]  Mark Steedman,et al.  Improving Dependency Parsers using Combinatory Categorial Grammar , 2014, EACL.

[88]  Timothy Dozat,et al.  Stanford’s Graph-based Neural Dependency Parser at the CoNLL 2017 Shared Task , 2017, CoNLL.

[89]  Yuan Zhang,et al.  Stack-propagation: Improved Representation Learning for Syntax , 2016, ACL.

[90]  Timothy Baldwin,et al.  Multilingual Deep Lexical Acquisition for HPSGs via Supertagging , 2006, EMNLP.

[91]  Wanxiang Che,et al.  The HIT-SCIR System for End-to-End Parsing of Universal Dependencies , 2017, CoNLL Shared Task.

[92]  Alexis Nasr,et al.  Revisiting Supertagging and Parsing: How to Use Supertags in Transition-Based Parsing , 2016, TAG.

[93]  Srinivas Bangalore,et al.  Supertagging: Using Complex Lexical Descriptions in Natural Language Processing , 2010 .