Transition-Based Parsing for Deep Dependency Structures

Derivations under different grammar formalisms allow extraction of various dependency structures. Particularly, bilexical deep dependency structures beyond surface tree representation can be derived from linguistic analysis grounded by CCG, LFG, and HPSG. Traditionally, these dependency structures are obtained as a by-product of grammar-guided parsers. In this article, we study the alternative data-driven, transition-based approach, which has achieved great success for tree parsing, to build general dependency graphs. We integrate existing tree parsing techniques and present two new transition systems that can generate arbitrary directed graphs in an incremental manner. Statistical parsers that are competitive in both accuracy and efficiency can be built upon these transition systems. Furthermore, the heterogeneous design of transition systems yields diversity of the corresponding parsing models and thus greatly benefits parser ensemble. Concerning the disambiguation problem, we introduce two new techniques, namely, transition combination and tree approximation, to improve parsing quality. Transition combination makes every action performed by a parser significantly change configurations. Therefore, more distinct features can be extracted for statistical disambiguation. With the same goal of extracting informative features, tree approximation induces tree backbones from dependency graphs and re-uses tree parsing techniques to produce tree-related features. We conduct experiments on CCG-grounded functor–argument analysis, LFG-grounded grammatical relation analysis, and HPSG-grounded semantic dependency analysis for English and Chinese. Experiments demonstrate that data-driven models with appropriate transition systems can produce high-quality deep dependency analysis, comparable to more complex grammar-driven models. Experiments also indicate the effectiveness of the heterogeneous design of transition systems for parser ensemble, transition combination, as well as tree approximation for statistical disambiguation.

[1]  Mark Steedman,et al.  CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank , 2007, CL.

[2]  M. Baltin,et al.  The Mental representation of grammatical relations , 1985 .

[3]  Mark Steedman,et al.  Building Deep Dependency Structures using a Wide-Coverage CCG Parser , 2002, ACL.

[4]  Michael Collins,et al.  Efficient Third-Order Dependency Parsers , 2010, ACL.

[5]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[6]  Jun'ichi Tsujii,et al.  Task-oriented Evaluation of Syntactic Parsers and Their Representations , 2008, ACL.

[7]  Joakim Nivre,et al.  Non-Projective Dependency Parsing in Expected Linear Time , 2009, ACL.

[8]  Joakim Nivre,et al.  Integrating Graph-Based and Transition-Based Dependency Parsers , 2008, ACL.

[9]  Stephen Clark,et al.  A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing , 2008, EMNLP.

[10]  Stephen Clark,et al.  Syntactic Processing Using the Generalized Perceptron and Beam Search , 2011, CL.

[11]  Weiwei Sun,et al.  Peking: Profiling Syntactic Tree Parsing Techniques for Semantic Graph Parsing , 2014, SemEval@COLING.

[12]  Jun'ichi Tsujii,et al.  Corpus-Oriented Grammar Development for Acquiring a Head-Driven Phrase Structure Grammar from the Penn Treebank , 2004, IJCNLP.

[13]  Dan Roth,et al.  The Importance of Syntactic Parsing and Inference in Semantic Role Labeling , 2008, CL.

[14]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[15]  Jun'ichi Tsujii,et al.  Feature Forest Models for Probabilistic HPSG Parsing , 2008, CL.

[16]  Stephen Clark,et al.  Shift-Reduce CCG Parsing with a Dependency Model , 2014, ACL.

[17]  Bernd Bohnet,et al.  Very high accuracy and fast dependency parsing is not a contradiction , 2010, COLING 2010.

[18]  Slav Petrov,et al.  Structured Training for Neural Network Transition-Based Parsing , 2015, ACL.

[19]  Ivan Titov,et al.  Online graph planarisation for synchronous parsing of semantic and syntactic dependencies , 2009, IJCAI 2009.

[20]  André F. T. Martins,et al.  Priberam: A Turbo Semantic Parser with Second Order Features , 2014, *SEMEVAL.

[21]  Joakim Nivre,et al.  Analyzing and Integrating Dependency Parsers , 2011, CL.

[22]  Heng Ji,et al.  Joint Event Extraction via Structured Prediction with Global Features , 2013, ACL.

[23]  Weiwei Sun,et al.  A Data-Driven, Factorization Parser for CCG Dependency Structures , 2015, ACL.

[24]  Eric P. Xing,et al.  Stacking Dependency Parsers , 2008, EMNLP.

[25]  Ivan A. Sag,et al.  Book Reviews: Head-driven Phrase Structure Grammar and German in Head-driven Phrase-structure Grammar , 1996, CL.

[26]  Ivan Titov,et al.  Multilingual Joint Parsing of Syntactic and Semantic Dependencies with a Latent Variable Model , 2013, CL.

[27]  Stephen Clark,et al.  Shift-Reduce CCG Parsing , 2011, ACL.

[28]  Stephan Oepen,et al.  Who Did What to Whom? A Contrastive Study of Syntacto-Semantic Dependencies , 2012, LAW@ACL.

[29]  Yusuke Miyao,et al.  Towards Framework-Independent Evaluation of Deep Linguistic Parsers , 2007 .

[30]  Mihai Surdeanu,et al.  Ensemble Models for Dependency Parsing: Cheap and Good? , 2010, HLT-NAACL.

[31]  FlickingerDan.  On building a more efficient grammar by exploiting types , 2000 .

[32]  Jun'ichi Tsujii,et al.  Efficient HPSG Parsing with Supertagging and CFG-Filtering , 2007, IJCAI.

[33]  Dan Flickinger,et al.  Minimal Recursion Semantics: An Introduction , 2005 .

[34]  Joakim Nivre,et al.  A Transition-Based Parser for 2-Planar Dependency Structures , 2010, ACL.

[35]  Chengqing Zong,et al.  A Minimum Error Weighting Combination Strategy for Chinese Semantic Role Labeling , 2010, COLING.

[36]  Adam Lopez,et al.  Training a Log-Linear Parser with Loss Functions via Softmax-Margin , 2011, EMNLP.

[37]  Martha Palmer,et al.  Getting the Most out of Transition-based Dependency Parsing , 2011, ACL.

[38]  Weiwei Sun,et al.  Peking: Building Semantic Dependency Graphs with a Hybrid Parser , 2015, SemEval@NAACL-HLT.

[39]  Joakim Nivre,et al.  Algorithms for Deterministic Incremental Dependency Parsing , 2008, CL.

[40]  Joakim Nivre,et al.  Divisible Transition Systems and Multiplanar Dependency Parsing , 2013, CL.

[41]  Alon Lavie,et al.  Parser Combination by Reparsing , 2006, NAACL.

[42]  James R. Curran,et al.  The Challenges of Parsing Chinese with Combinatory Categorial Grammar , 2012, HLT-NAACL.

[43]  Mary P. Harper,et al.  Self-Training with Products of Latent Variable Grammars , 2010, EMNLP.

[44]  Joakim Nivre,et al.  Transition-based Dependency Parsing with Rich Non-local Features , 2011, ACL.

[45]  Stephan Oepen,et al.  Discriminant-Based MRS Banking , 2006, LREC.

[46]  Xavier Carreras,et al.  An Empirical Study of Semi-supervised Structured Conditional Models for Dependency Parsing , 2009, EMNLP.

[47]  Kenji Sagae,et al.  Dynamic Programming for Linear-Time Incremental Parsing , 2010, ACL.

[48]  Brian Roark,et al.  Incremental Parsing with the Perceptron Algorithm , 2004, ACL.

[49]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.

[50]  Fernando Pereira,et al.  Discriminative learning and spanning tree algorithms for dependency parsing , 2006 .

[51]  Jun'ichi Tsujii,et al.  Shift-Reduce Dependency DAG Parsing , 2008, COLING.

[52]  Weiwei Sun,et al.  Data-driven, PCFG-based and Pseudo-PCFG-based Models for Chinese Dependency Parsing , 2013, Transactions of the Association for Computational Linguistics.

[53]  James R. Curran,et al.  Chinese CCGbank: extracting CCG derivations from the Penn Chinese Treebank , 2010, COLING.

[54]  Weiwei Sun,et al.  Grammatical Relations in Chinese: GB-Ground Extraction and Data-Driven Parsing , 2014, ACL.

[55]  Haizhou Li,et al.  K-Best Combination of Syntactic Parsers , 2009, EMNLP.

[56]  Yuji Matsumoto,et al.  Statistical Dependency Analysis with Support Vector Machines , 2003, IWPT.

[57]  Fernando Pereira,et al.  Online Learning of Approximate Dependency Parsing Algorithms , 2006, EACL.

[58]  Daniel Gildea,et al.  Plurality, Negation, and Quantification: Towards Comprehensive Quantifier Scope Disambiguation , 2013, ACL.

[59]  Richard Johansson,et al.  The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages , 2009, CoNLL Shared Task.

[60]  Adam Lopez,et al.  A Comparison of Loopy Belief Propagation and Dual Decomposition for Integrated CCG Supertagging and Parsing , 2011, ACL.

[61]  Mark Steedman,et al.  An Incremental Algorithm for Transition-based CCG Parsing , 2015, NAACL.

[62]  Mark Steedman,et al.  Large-scale Semantic Parsing without Question-Answer Pairs , 2014, TACL.

[63]  Michael A. Covington,et al.  A Fundamental Algorithm for Dependency Parsing , 2004 .

[64]  Joakim Nivre,et al.  Pseudo-Projective Dependency Parsing , 2005, ACL.

[65]  Richard Johansson,et al.  The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies , 2008, CoNLL.

[66]  Eric P. Xing,et al.  Concise Integer Linear Programming Formulations for Dependency Parsing , 2009, ACL.

[67]  Bernd Bohnet,et al.  Top Accuracy and Fast Dependency Parsing is not a Contradiction , 2010, COLING.

[68]  James R. Curran,et al.  Wide-Coverage Efficient Statistical Parsing with CCG and Log-Linear Models , 2007, Computational Linguistics.

[69]  Martin Kay,et al.  Syntactic Process , 1979, ACL.