Multitask Easy-First Dependency Parsing: Exploiting Complementarities of Different Dependency Representations

We present a parsing model for projective dependency trees which takes advantage of the existence of complementary dependency annotations for a language. This is the case for Arabic with the availability of CATiB and UD treebanks. Our system performs syntactic parsing according to both annotation types jointly as a sequence of arc-creating operations following the Easy-First approach, and partially created trees for one annotation type are also available to the other as features for the score function. This method gives error reduction of 9.9% on CATiB and 6.1% on UD compared to a single-task baseline, and ablation tests show that the main contribution of this reduction is given by sharing tree representation between tasks, and not simply sharing BiLSTM layers as is usually performed in NLP multitask systems.

[1]  Noah A. Smith,et al.  Learning Joint Semantic Parsers from Disjoint Data , 2018, NAACL.

[2]  Nizar Habash,et al.  Introduction to Arabic Natural Language Processing , 2010, Introduction to Arabic Natural Language Processing.

[3]  Makoto Miwa,et al.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures , 2016, ACL.

[4]  Yuji Matsumoto,et al.  Adversarial Training for Cross-Domain Universal Dependency Parsing , 2017, CoNLL Shared Task.

[5]  Noah A. Smith,et al.  One Parser, Many Languages , 2016, ArXiv.

[6]  Joseph Le Roux,et al.  Deep Lexical Segmentation and Syntactic Parsing in the Easy-First Dependency Framework , 2016, HLT-NAACL.

[7]  Anders Søgaard,et al.  Deep multi-task learning with low level tasks supervised at lower layers , 2016, ACL.

[8]  Joakim Nivre,et al.  Training Deterministic Parsers with Non-Deterministic Oracles , 2013, TACL.

[9]  Nathan Schneider,et al.  Made for Each Other: Broad-Coverage Semantic Structures Meet Preposition Supersenses , 2019, CoNLL.

[10]  M. Maamouri,et al.  The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus , 2004 .

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  Alexander Koller,et al.  Compositional Semantic Parsing across Graphbanks , 2019, ACL.

[13]  Joakim Nivre,et al.  Parser Training with Heterogeneous Treebanks , 2018, ACL.

[14]  Nizar Habash,et al.  LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual , 2013, ArXiv.

[15]  Anders Søgaard,et al.  Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies , 2019, ACL.

[16]  Ari Rappoport,et al.  Multitask Parsing Across Semantic Representations , 2018, ACL.

[17]  Eliyahu Kiperwasser,et al.  Easy-First Dependency Parsing with Hierarchical Tree LSTMs , 2016, TACL.

[18]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[19]  Yusuke Miyao,et al.  SemEval 2015 Task 18: Broad-Coverage Semantic Dependency Parsing , 2015, *SEMEVAL.

[20]  Nizar Habash,et al.  CamelParser: A system for Arabic Syntactic Analysis and Morphological Disambiguation , 2016, COLING.

[21]  Nizar Habash,et al.  Dependency Parsing of Modern Standard Arabic with Lexical and Inflectional Features , 2013, CL.

[22]  Nizar Habash,et al.  Universal Dependencies for Arabic , 2017, WANLP@EACL.

[23]  Nizar Habash,et al.  CATiB: The Columbia Arabic Treebank , 2009, ACL.

[24]  Jürgen Schmidhuber,et al.  Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition , 2005, ICANN.

[25]  Nizar Habash,et al.  Improving Arabic Diacritization through Syntactic Analysis , 2015, EMNLP.

[26]  Ido Dagan,et al.  Semantics as a Foreign Language , 2018, EMNLP.

[27]  Yoshimasa Tsuruoka,et al.  A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks , 2016, EMNLP.

[28]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[29]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[30]  Joakim Nivre,et al.  MaltParser: A Data-Driven Parser-Generator for Dependency Parsing , 2006, LREC.

[31]  Noah A. Smith,et al.  Deep Multitask Learning for Semantic Dependency Parsing , 2017, ACL.

[32]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.