Seq2seq Dependency Parsing

This paper presents a sequence to sequence (seq2seq) dependency parser by directly predicting the relative position of head for each given word, which therefore results in a truly end-to-end seq2seq dependency parser for the first time. Enjoying the advantage of seq2seq modeling, we enrich a series of embedding enhancement, including firstly introduced subword and node2vec augmentation. Meanwhile, we propose a beam search decoder with tree constraint and subroot decomposition over the sequence to furthermore enhance our seq2seq parser. Our parser is evaluated on benchmark treebanks, being on par with the state-of-the-art parsers by achieving 94.11% UAS on PTB and 88.78% UAS on CTB, respectively.

[1]  Hai Zhao,et al.  Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification , 2017, ACL.

[2]  Hao Zhang,et al.  Generalized Higher-Order Dependency Parsing with Cube Pruning , 2012, EMNLP.

[3]  Baobao Chang,et al.  Graph-based Dependency Parsing with Bidirectional LSTM , 2016, ACL.

[4]  Christopher D. Manning,et al.  Generating Typed Dependency Parses from Phrase Structure Parses , 2006, LREC.

[5]  Hai Zhao,et al.  Deep Enhanced Representation for Implicit Discourse Relation Recognition , 2018, COLING.

[6]  Noah A. Smith,et al.  Turning on the Turbo: Fast Third-Order Non-Projective Turbo Parsers , 2013, ACL.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  Hai Zhao,et al.  Probabilistic Graph-based Dependency Parsing with Convolutional Neural Network , 2016, ACL.

[9]  Hai Zhao,et al.  Neural Character-level Dependency Parsing for Chinese , 2018, AAAI.

[10]  Noah A. Smith,et al.  What Do Recurrent Neural Network Grammars Learn About Syntax? , 2016, EACL.

[11]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[12]  Yue Zhang,et al.  A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing , 2015, ACL.

[13]  Joakim Nivre,et al.  An Efficient Algorithm for Projective Dependency Parsing , 2003, IWPT.

[14]  Hai Zhao,et al.  Modeling Multi-turn Conversation with Deep Utterance Aggregation , 2018, COLING.

[15]  Hai Zhao,et al.  Fourth-Order Dependency Parsing , 2012, COLING.

[16]  Xuanjing Huang,et al.  A Re-ranking Model for Dependency Parser with Recursive Convolutional Neural Network , 2015, ACL.

[17]  Enhong Chen,et al.  Stack-based Multi-layer Attention for Transition-based Dependency Parsing , 2017, EMNLP.

[18]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[19]  Hai Zhao,et al.  A Full End-to-End Semantic Role Labeler, Syntactic-agnostic Over Syntactic-aware? , 2018, COLING.

[20]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[21]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[22]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[23]  Jason Eisner.  Efficient Normal-Form Parsing for Combinatory Categorial Grammar , 1996, ACL.

[24]  Alexander M. Rush,et al.  OpenNMT: Open-Source Toolkit for Neural Machine Translation , 2017, ACL.

[25]  Hai Zhao,et al.  A Transition-based System for Universal Dependency Parsing , 2017, CoNLL Shared Task.

[26]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[27]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[28]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[29]  Mirella Lapata,et al.  Dependency Parsing as Head Selection , 2016, EACL.

[30]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[31]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[32]  Alexander M. Rush,et al.  Sequence-to-Sequence Learning as Beam-Search Optimization , 2016, EMNLP.

[33]  Joakim Nivre,et al.  Integrating Graph-Based and Transition-Based Dependency Parsers , 2008, ACL.

[34]  Stephen Clark,et al.  A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing , 2008, EMNLP.

[35]  Hai Zhao,et al.  Syntax for Semantic Role Labeling, To Be, Or Not To Be , 2018, ACL.

[36]  Timothy Dozat,et al.  Deep Biaffine Attention for Neural Dependency Parsing , 2016, ICLR.

[37]  Hai Zhao,et al.  Moon IME: Neural-based Chinese Pinyin Aided Input Method with Customizable Association , 2018, ACL.

[38]  Koby Crammer,et al.  Online Large-Margin Training of Dependency Parsers , 2005, ACL.

[39]  Luke S. Zettlemoyer,et al.  AllenNLP: A Deep Semantic Natural Language Processing Platform , 2018, ArXiv.

[40]  Slav Petrov,et al.  Globally Normalized Transition-Based Neural Networks , 2016, ACL.

[41]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[42]  Slav Petrov,et al.  Structured Training for Neural Network Transition-Based Parsing , 2015, ACL.

[43]  Hai Zhao,et al.  Pair-Aware Neural Sentence Modeling for Implicit Discourse Relation Classification , 2017, IEA/AIE.

[44]  Eliyahu Kiperwasser,et al.  Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations , 2016, TACL.

[45]  Hao Zhang,et al.  Enforcing Structural Diversity in Cube-pruned Dependency Parsing , 2014, ACL.