Generalized Higher-Order Dependency Parsing with Cube Pruning

State-of-the-art graph-based parsers use features over higher-order dependencies that rely on decoding algorithms that are slow and difficult to generalize. On the other hand, transition-based dependency parsers can easily utilize such features without increasing the linear complexity of the shift-reduce system beyond a constant. In this paper, we attempt to address this imbalance for graph-based parsing by generalizing the Eisner (1996) algorithm to handle arbitrary features over higher-order dependencies. The generalization is at the cost of asymptotic efficiency. To account for this, cube pruning for decoding is utilized (Chiang, 2007). For the first time, label tuple and structural features such as valencies can be scored efficiently with third-order features in a graph-based parser. Our parser achieves the state-of-art unlabeled accuracy of 93.06% and labeled accuracy of 91.86% on the standard test set for English, at a faster speed than a reimplementation of the third-order model of Koo et al. (2010).

[1]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.

[2]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[3]  Eugene Charniak,et al.  Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking , 2005, ACL.

[4]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[5]  Sabine Buchholz,et al.  CoNLL-X Shared Task on Multilingual Dependency Parsing , 2006, CoNLL.

[6]  Sebastian Riedel,et al.  Incremental Integer Linear Programming for Non-projective Dependency Parsing , 2006, EMNLP.

[7]  Christopher D. Manning,et al.  Generating Typed Dependency Parses from Phrase Structure Parses , 2006, LREC.

[8]  Fernando Pereira,et al.  Online Learning of Approximate Dependency Parsing Algorithms , 2006, EACL.

[9]  David Chiang,et al.  Hierarchical Phrase-Based Translation , 2007, CL.

[10]  Bo Xu,et al.  Probabilistic Parsing Action Models for Multi-Lingual Dependency Parsing , 2007, EMNLP.

[11]  Sebastian Riedel,et al.  The CoNLL 2007 Shared Task on Dependency Parsing , 2007, EMNLP.

[12]  Richard Johansson,et al.  Incremental Dependency Parsing Using Online Learning , 2007, EMNLP-CoNLL.

[13]  Keith Hall,et al.  K-best Spanning Tree Parsing , 2007, ACL.

[14]  Tetsuji Nakagawa,et al.  Multilingual Dependency Parsing Using Global Features , 2007, EMNLP.

[15]  Dan Klein,et al.  Improved Inference for Unlexicalized Parsing , 2007, NAACL.

[16]  Xavier Carreras,et al.  Experiments with a Higher-Order Projective Dependency Parser , 2007, EMNLP.

[17]  Ivan Titov,et al.  Fast and Robust Multilingual Dependency Parsing with a Generative Latent Variable Model , 2007, EMNLP.

[18]  Joakim Nivre,et al.  Characterizing the Errors of Data-Driven Dependency Parsing Models , 2007, EMNLP.

[19]  Brian Roark,et al.  Classifying Chart Cells for Quadratic Complexity Context-Free Inference , 2008, COLING.

[20]  Stephen Clark,et al.  A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing , 2008, EMNLP.

[21]  Joakim Nivre,et al.  Integrating Graph-Based and Transition-Based Dependency Parsers , 2008, ACL.

[22]  Liang Huang,et al.  Forest Reranking: Discriminative Parsing with Non-Local Features , 2008, ACL.

[23]  David A. Smith,et al.  Dependency Parsing by Belief Propagation , 2008, EMNLP.

[24]  Giorgio Satta,et al.  Treebank Grammar Techniques for Non-Projective Dependency Parsing , 2009, EACL.

[25]  Brian Roark,et al.  Linear Complexity Context-Free Parsing Pipelines via Chart Constraints , 2009, NAACL.

[26]  Eric P. Xing,et al.  Concise Integer Linear Programming Formulations for Dependency Parsing , 2009, ACL.

[27]  Giorgio Satta,et al.  Optimal Reduction of Rule Length in Linear Context-Free Rewriting Systems , 2009, NAACL.

[28]  Noah A. Smith,et al.  Cube Summing, Approximate Inference with Non-Local Features, and Dynamic Programming without Semirings , 2009, EACL.

[29]  Kenji Sagae,et al.  Dynamic Programming for Linear-Time Incremental Parsing , 2010, ACL.

[30]  Ben Taskar,et al.  Structured Prediction Cascades , 2010, AISTATS.

[31]  Eric P. Xing,et al.  Turbo Parsers: Dependency Parsing by Approximate Variational Inference , 2010, EMNLP.

[32]  Michael Collins,et al.  Efficient Third-Order Dependency Parsers , 2010, ACL.

[33]  Giorgio Satta,et al.  Efficient Parsing of Well-Nested Linear Context-Free Rewriting Systems , 2010, HLT-NAACL.

[34]  Alexander M. Rush,et al.  Dual Decomposition for Parsing with Non-Projective Head Automata , 2010, EMNLP.

[35]  Noah A. Smith,et al.  Dual Decomposition with Many Overlapping Components , 2011, EMNLP.

[36]  Joakim Nivre,et al.  Transition-based Dependency Parsing with Rich Non-local Features , 2011, ACL.

[37]  Brian Roark,et al.  Beam-Width Prediction for Efficient Context-Free Parsing , 2011, ACL.

[38]  Alexander M. Rush,et al.  Vine Pruning for Efficient Multi-Pass Dependency Parsing , 2012, NAACL.

[39]  Yang Guo,et al.  Structured Perceptron with Inexact Search , 2012, NAACL.