In-Order Transition-based Constituent Parsing

Both bottom-up and top-down strategies have been used for neural transition-based constituent parsing. The parsing strategies differ in terms of the order in which they recognize productions in the derivation tree, where bottom-up strategies and top-down strategies take post-order and pre-order traversal over trees, respectively. Bottom-up parsers benefit from rich features from readily built partial parses, but lack lookahead guidance in the parsing process; top-down parsers benefit from non-local guidance for local decisions, but rely on a strong encoder over the input to predict a constituent hierarchy before its construction. To mitigate both issues, we propose a novel parsing system based on in-order traversal over syntactic trees, designing a set of transition actions to find a compromise between bottom-up constituent information and top-down lookahead information. Based on stack-LSTM, our psycholinguistically motivated constituent parsing system achieves 91.8 F1 on the WSJ benchmark. Furthermore, the system achieves 93.6 F1 with supervised reranking and 94.2 F1 with semi-supervised reranking, which are the best results on the WSJ benchmark.

[1]  James Cross,et al.  Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles , 2016, EMNLP.

[2]  William Schuler,et al.  An Analysis of Frequency- and Memory-Based Processing Costs , 2013, NAACL.

[3]  Andrew Y. Ng,et al.  Parsing with Compositional Vector Grammars , 2013, ACL.

[4]  Hiroyuki Shindo,et al.  Bayesian Symbol-Refined Tree Substitution Grammars for Syntactic Parsing , 2012, ACL.

[5]  Fei Xia,et al.  The Penn Chinese TreeBank: Phrase structure annotation of a large corpus , 2005, Natural Language Engineering.

[6]  Noah A. Smith,et al.  Training with Exploration Improves a Greedy Stack LSTM Parser , 2016, EMNLP.

[7]  Mark Steedman,et al.  The syntactic process , 2004, Language, speech, and communication.

[8]  Eliyahu Kiperwasser,et al.  Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations , 2016, TACL.

[9]  Michael Collins,et al.  Head-Driven Statistical Models for Natural Language Parsing , 2003, CL.

[10]  Luke S. Zettlemoyer,et al.  LSTM CCG Parsing , 2016, NAACL.

[11]  Mark Johnson,et al.  PCFG Models of Linguistic Tree Representations , 1998, CL.

[12]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[13]  Stephen Clark,et al.  Transition-Based Parsing of the Chinese Treebank using a Global Discriminative Model , 2009, IWPT.

[14]  Nianwen Xue,et al.  Feature Optimization for Constituent Parsing via Neural Networks , 2015, ACL.

[15]  Dan Klein,et al.  Neural CRF Parsing , 2015, ACL.

[16]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[17]  Wenduan Xu,et al.  LSTM Shift-Reduce CCG Parsing , 2016, EMNLP.

[18]  Yue Zhang,et al.  Shift-Reduce Constituent Parsing with Neural Lookahead Features , 2016, TACL.

[19]  Brian Roark,et al.  Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing , 2009, EMNLP.

[20]  Jianfeng Gao,et al.  Bi-directional Attention with Agreement for Dependency Parsing , 2016, EMNLP.

[21]  Liang Huang,et al.  Forest Reranking: Discriminative Parsing with Non-Local Features , 2008, ACL.

[22]  Daniel J. Rosenkrantz,et al.  Deterministic Left Corner Parsing (Extended Abstract) , 1970, SWAT.

[23]  Noah A. Smith,et al.  Recurrent Neural Network Grammars , 2016, NAACL.

[24]  Taro Watanabe,et al.  Transition-based Neural Constituent Parsing , 2015, ACL.

[25]  William Schuler,et al.  Broad-Coverage Parsing Using Human-Like Memory Constraints , 2010, CL.

[26]  Eugene Charniak,et al.  Parsing as Language Modeling , 2016, EMNLP.

[27]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[28]  Geoffrey E. Hinton,et al.  Grammar as a Foreign Language , 2014, NIPS.

[29]  Alon Lavie,et al.  A Classifier-Based Parser with Linear Run-Time Complexity , 2005, IWPT.

[30]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[31]  Slav Petrov,et al.  Globally Normalized Transition-Based Neural Networks , 2016, ACL.

[32]  Martin Kay,et al.  Syntactic Process , 1979, ACL.

[33]  Nianwen Xue,et al.  Joint POS Tagging and Transition-based Constituent Parsing in Chinese with Non-local Features , 2014, ACL.

[34]  Mark Johnson,et al.  Robust probabilistic predictive syntactic processing: motivations, models, and applications , 2001 .

[35]  Yue Zhang,et al.  Fast and Accurate Shift-Reduce Constituent Parsing , 2013, ACL.

[36]  Eugene Charniak,et al.  Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking , 2005, ACL.

[37]  Noah A. Smith,et al.  What Do Recurrent Neural Network Grammars Learn About Syntax? , 2016, EACL.

[38]  Brian Roark,et al.  Efficient probabilistic top-down and left-corner parsing , 1999, ACL.

[39]  Eugene Charniak,et al.  A Maximum-Entropy-Inspired Parser , 2000, ANLP.