Span-Based LCFRS-2 Parsing

The earliest models for discontinuous constituency parsers used mildly context-sensitive grammars, but the fashion has changed in recent years to grammar-less transition-based parsers that use strong neural probabilistic models to greedily predict transitions. We argue that grammar-based approaches still have something to contribute on top of what is offered by transition-based parsers. Concretely, by using a grammar formalism to restrict the space of possible trees we can use dynamic programming parsing algorithms for exact search for the most probable tree. Previous chart-based parsers for discontinuous formalisms used probabilistically weak generative models. We instead use a span-based discriminative neural model that preserves the dynamic programming properties of the chart parsers. Our parser does not use an explicit grammar, but it does use explicit grammar formalism constraints: we generate only trees that are within the LCFRS-2 formalism. These properties allow us to construct a new parsing algorithm that runs in lower worst-case time complexity of O(l nˆ4 +nˆ6), where n is the sentence length and l is the number of unique non-terminal labels. This parser is efficient in practice, provides best results among chart-based parsers, and is competitive with the best transition based parsers. We also show that the main bottleneck for further improvement in performance is in the restriction of fan-out to degree 2. We show that well-nestedness is helpful in speeding up parsing, but lowers accuracy.

[1]  André F. T. Martins,et al.  Parsing as Reduction , 2015, ACL.

[2]  Tobias Denkinger,et al.  Implementation of a Chomsky-Schützenberger n-best parser for weighted multiple context-free grammars , 2019, NAACL.

[3]  Sampo Pyysalo,et al.  Universal Dependencies v1: A Multilingual Treebank Collection , 2016, LREC.

[4]  Marco Kuhlmann,et al.  Mildly Context-Sensitive Dependency Languages , 2007, ACL.

[5]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[6]  Laura Kallmeyer,et al.  PLCFRS Parsing of English Discontinuous Constituents , 2011, IWPT.

[7]  Milos Stanojevic,et al.  Neural Discontinuous Constituency Parsing , 2017, EMNLP.

[8]  Shay B. Cohen,et al.  Unlexicalized Transition-based Discontinuous Constituency Parsing , 2019, Transactions of the Association for Computational Linguistics.

[9]  James Cross,et al.  Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles , 2016, EMNLP.

[10]  Laura Kallmeyer,et al.  Parsing Beyond Context-Free Grammars , 2010, Cognitive Technologies.

[11]  David A. McAllester.  On the complexity analysis of static analyses , 1999, JACM.

[12]  Alexander Koller,et al.  Dependency Trees and the Strong Generative Capacity of CCG , 2009, EACL.

[13]  Mark-Jan Nederhof,et al.  Squibs and Discussions: Weighted Deductive Parsing and Knuth’s Algorithm , 2003, CL.

[14]  Carlos G'omez-Rodr'iguez,et al.  Discontinuous Constituent Parsing with Pointer Networks , 2020, AAAI.

[15]  Laura Kallmeyer.  Linear Context-Free Rewriting Systems , 2013, Lang. Linguistics Compass.

[16]  Giorgio Satta,et al.  Efficient Parsing of Well-Nested Linear Context-Free Rewriting Systems , 2010, HLT-NAACL.

[17]  Maximin Coavoux,et al.  Incremental Discontinuous Phrase Structure Parsing with the GAP Transition , 2017, EACL.

[18]  Joseph Le Roux,et al.  Efficient Discontinuous Phrase-Structure Parsing via the Generalized Maximum Spanning Arborescence , 2017, EMNLP.

[19]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[20]  David J. Weir,et al.  Characterizing Structural Descriptions Produced by Various Grammatical Formalisms , 1987, ACL.

[21]  Dan Klein,et al.  Multilingual Constituency Parsing with Self-Attention and Pre-Training , 2018, ACL.

[22]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[23]  Aravind K. Joshi,et al.  Natural language parsing: Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? , 1985 .

[24]  Aravind K. Joshi,et al.  Unavoidable Ill-nestedness in Natural Language and the Adequacy of Tree Local-MCTAG Induced Dependency Structures , 2010, TAG.

[25]  Yannick Versley.  Discontinuity (Re)²-visited: A Minimalist Approach to Pseudoprojective Constituent Parsing , 2016 .

[26]  Wolfgang Maier,et al.  Discontinuous Incremental Shift-reduce Parsing , 2015, ACL.

[27]  Kevin Duh,et al.  DyNet: The Dynamic Neural Network Toolkit , 2017, ArXiv.

[28]  Shay B. Cohen,et al.  Discontinuous Constituency Parsing with a Stack-Free Transition System and a Dynamic Oracle , 2019, NAACL.

[29]  Dan Klein,et al.  Constituency Parsing with a Self-Attentive Encoder , 2018, ACL.

[30]  Jürgen Schmidhuber,et al.  Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition , 2005, ICANN.

[31]  Joakim Nivre,et al.  Mildly Non-Projective Dependency Structures , 2006, ACL.

[32]  Ben Taskar,et al.  Max-Margin Markov Networks , 2003, NIPS.

[33]  Daniel Götzmann.  Multiple Context-Free Grammars , 2007 .

[34]  Rens Bod,et al.  Data-Oriented Parsing with Discontinuous Constituents and Function Tags , 2016, J. Lang. Model..

[35]  Timm Lichte,et al.  Characterizing Discontinuity in Constituent Treebanks , 2009, FG.

[36]  Stuart M. Shieber,et al.  Principles and Implementation of Deductive Parsing , 1994, J. Log. Program..

[37]  Dan Klein,et al.  What’s Going On in Neural Constituency Parsers? An Analysis , 2018, NAACL.

[38]  Dan Klein,et al.  A Minimal Span-Based Neural Constituency Parser , 2017, ACL.

[39]  Ben Taskar,et al.  Max-Margin Parsing , 2004, EMNLP.

[40]  Rens Bod,et al.  Discontinuous Parsing with an Efficient and Accurate DOP Model , 2013, IWPT.

[41]  Wolfgang Lezius,et al.  TIGER: Linguistic Interpretation of a German Corpus , 2004 .

[42]  Laura Kallmeyer,et al.  PLCFRS Parsing Revisited: Restricting the Fan-Out to Two , 2012, TAG.