Discontinuous Constituency Parsing with a Stack-Free Transition System and a Dynamic Oracle

We introduce a novel transition system for discontinuous constituency parsing. Instead of storing subtrees in a stack --i.e. a data structure with linear-time sequential access-- the proposed system uses a set of parsing items, with constant-time random access. This change makes it possible to construct any discontinuous constituency tree in exactly $4n - 2$ transitions for a sentence of length $n$. At each parsing step, the parser considers every item in the set to be combined with a focus item and to construct a new constituent in a bottom-up fashion. The parsing strategy is based on the assumption that most syntactic structures can be parsed incrementally and that the set --the memory of the parser-- remains reasonably small on average. Moreover, we introduce a provably correct dynamic oracle for the new transition system, and present the first experiments in discontinuous constituency parsing using a dynamic oracle. Our parser obtains state-of-the-art results on three English and German discontinuous treebanks.

[1]  Laura Kallmeyer,et al.  Parsing Beyond Context-Free Grammars , 2010, Cognitive Technologies.

[2]  Milos Stanojevic,et al.  Neural Discontinuous Constituency Parsing , 2017, EMNLP.

[3]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[4]  Wolfgang Maier,et al.  Discontinuous Incremental Shift-reduce Parsing , 2015, ACL.

[5]  Shay B. Cohen,et al.  Unlexicalized Transition-based Discontinuous Constituency Parsing , 2019, Transactions of the Association for Computational Linguistics.

[6]  Giorgio Satta,et al.  Elimination of Spurious Ambiguity in Transition-Based Dependency Parsing , 2012, ArXiv.

[7]  Carlos Gómez-Rodríguez,et al.  Dynamic Oracles for Top-Down and In-Order Shift-Reduce Constituent Parsing , 2018, EMNLP.

[8]  Wojciech Skut,et al.  An Annotation Scheme for Free Word Order Languages , 1997, ANLP.

[9]  Quoc V. Le,et al.  Adding Gradient Noise Improves Learning for Very Deep Networks , 2015, ArXiv.

[10]  Carlos Gómez-Rodríguez,et al.  Non-Projective Dependency Parsing with Non-Local Transitions , 2017, NAACL.

[11]  Joakim Nivre,et al.  Training Deterministic Parsers with Non-Deterministic Oracles , 2013, TACL.

[12]  Dan Klein,et al.  Neural CRF Parsing , 2015, ACL.

[13]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[14]  James Cross,et al.  Incremental Parsing with Minimal Features Using Bi-Directional LSTM , 2016, ACL.

[15]  Yannick Versley.  Discontinuity (Re)²-visited: A Minimalist Approach to Pseudoprojective Constituent Parsing , 2016 .

[16]  Wolfgang Lezius,et al.  TIGER: Linguistic Interpretation of a German Corpus , 2004 .

[17]  Joakim Nivre,et al.  Non-Projective Dependency Parsing in Expected Linear Time , 2009, ACL.

[18]  Maximin Coavoux,et al.  Incremental Discontinuous Phrase Structure Parsing with the GAP Transition , 2017, EACL.

[19]  Joakim Nivre,et al.  A Dynamic Oracle for Arc-Eager Dependency Parsing , 2012, COLING.

[20]  André F. T. Martins,et al.  Parsing as Reduction , 2015, ACL.

[21]  Frank Keller,et al.  Probabilistic Parsing for German Using Sister-Head Dependencies , 2003, ACL.

[22]  Giuseppe Attardi,et al.  Experiments with a Multilanguage Non-Projective Dependency Parser , 2006, CoNLL.

[23]  Maximin Coavoux,et al.  Neural Greedy Constituent Parsing with Dynamic Oracles , 2016, ACL.

[24]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[25]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[26]  Rens Bod,et al.  Data-Oriented Parsing with Discontinuous Constituents and Function Tags , 2016, J. Lang. Model..

[27]  Carlos Gómez-Rodríguez,et al.  An Efficient Dynamic Oracle for Unrestricted Non-Projective Parsing , 2015, ACL.

[28]  Timm Lichte,et al.  Discontinuous parsing with continuous trees , 2016 .

[29]  Benoît Crabbé,et al.  Multilingual discriminative lexicalized phrase structure parsing , 2015, EMNLP.

[30]  Dan Klein,et al.  Less Grammar, More Features , 2014, ACL.

[31]  Dan Klein,et al.  A Minimal Span-Based Neural Constituency Parser , 2017, ACL.

[32]  Maximin Coavoux,et al.  Multilingual Lexicalized Constituency Parsing with Word-Level Auxiliary Tasks , 2017, EACL.

[33]  David J. Weir,et al.  Characterizing Structural Descriptions Produced by Various Grammatical Formalisms , 1987, ACL.

[34]  Giorgio Satta,et al.  A Polynomial-Time Dynamic Oracle for Non-Projective Dependency Parsing , 2014, EMNLP.

[35]  Joseph Le Roux,et al.  Efficient Discontinuous Phrase-Structure Parsing via the Generalized Maximum Spanning Arborescence , 2017, EMNLP.

[36]  Kilian Gebhardt,et al.  Generic refinement of expressive grammar formalisms with an application to discontinuous constituent parsing , 2018, COLING.

[37]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[38]  Carlos Gómez-Rodríguez,et al.  Faster Shift-Reduce Constituent Parsing with a Non-Binary, Bottom-Up Strategy , 2018, Artif. Intell..

[39]  James Cross,et al.  Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles , 2016, EMNLP.

[40]  Nizar Habash,et al.  Overview of the SPMRL 2013 Shared Task: A Cross-Framework Evaluation of Parsing Morphologically Rich Languages , 2013, SPMRL@EMNLP.

[41]  Yannick Versley.  Incorporating Semi-supervised Features into Discontinuous Easy-First Constituent Parsing , 2014, ArXiv.

[42]  Yannick Versley,et al.  Experiments with Easy-first nonprojective constituent parsing , 2014 .

[43]  Eliyahu Kiperwasser,et al.  Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations , 2016, TACL.

[44]  Laura Kallmeyer,et al.  PLCFRS Parsing of English Discontinuous Constituents , 2011, IWPT.

[45]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[46]  Noah A. Smith,et al.  Training with Exploration Improves a Greedy Stack LSTM Parser , 2016, EMNLP.

[47]  Christopher D. Manning,et al.  Arc-swift: A Novel Transition System for Dependency Parsing , 2017, ACL.