Project-then-Transfer: Effective Two-stage Cross-lingual Transfer for Semantic Dependency Parsing

This paper describes the first report on cross-lingual transfer for semantic dependency parsing. We present the insight that there are twodifferent kinds of cross-linguality, namely sur-face level and mantic level, and try to cap-ture both kinds of cross-linguality by combin-ing annotation projection and model transferof pre-trained language models. Our exper-iments showed that the performance of our graph-based semantic dependency parser almost achieved the approximated upper bound.

[1]  Noah A. Smith,et al.  Learning Joint Semantic Parsers from Disjoint Data , 2018, NAACL.

[2]  Noah A. Smith,et al.  Many Languages, One Parser , 2016, TACL.

[3]  Isabelle Augenstein,et al.  Parameter sharing between dependency parsers for related languages , 2018, EMNLP.

[4]  Mohammad Sadegh Rasooli,et al.  Multitask Learning for Cross-Lingual Transfer of Broad-coverage Semantic Dependencies , 2020, EMNLP.

[5]  Stephan Oepen,et al.  MRP 2019: Cross-Framework Meaning Representation Parsing , 2019, CoNLL.

[6]  Martin Potthast,et al.  CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies , 2018, CoNLL.

[7]  Yee Leung,et al.  CUHK at MRP 2019: Transition-Based Parser with Cross-Framework Variable-Arity Resolve Action , 2019, CoNLL.

[8]  Hiroaki Ozaki,et al.  Hitachi at MRP 2019: Unified Encoder-to-Biaffine Network for Cross-Framework Meaning Representation Parsing , 2019, CoNLL Shared Task.

[9]  Ari Rappoport,et al.  Multitask Parsing Across Semantic Representations , 2018, ACL.

[10]  Nizar Habash,et al.  CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies , 2017, CoNLL.

[11]  Stephan Oepen,et al.  SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing , 2014, *SEMEVAL.

[12]  Masao Utiyama,et al.  SJTU-NICT at MRP 2019: Multi-Task Learning for End-to-End Uniform Semantic Graph Parsing , 2019, CoNLL.

[13]  Min Zhang,et al.  SUDA-Alibaba at MRP 2019: Graph-Based Models with BERT , 2019, CoNLL Shared Task.

[14]  Danqi Chen,et al.  of the Association for Computational Linguistics: , 2001 .

[15]  Yusuke Miyao,et al.  SemEval 2015 Task 18: Broad-Coverage Semantic Dependency Parsing , 2015, *SEMEVAL.

[16]  Anders Søgaard,et al.  Multi-Task Semantic Dependency Parsing with Policy Gradient for Learning Easy-First Strategies , 2019, ACL.

[17]  Noah A. Smith,et al.  Deep Multitask Learning for Semantic Dependency Parsing , 2017, ACL.

[18]  Stephan Oepen,et al.  Towards Comparability of Linguistic Graph Banks for Semantic Parsing , 2016, LREC.

[19]  Sampo Pyysalo,et al.  Universal Dependencies v1: A Multilingual Treebank Collection , 2016, LREC.

[20]  David Yarowsky,et al.  Cross-lingual Dependency Parsing Based on Distributed Representations , 2015, ACL.

[21]  Ronan Collobert,et al.  Neural Network-based Word Alignment through Score Aggregation , 2016, WMT.

[22]  Shay B. Cohen,et al.  Cross-Lingual Abstract Meaning Representation Parsing , 2017, NAACL.

[23]  Daniel Kondratyuk,et al.  75 Languages, 1 Model: Parsing Universal Dependencies Universally , 2019, EMNLP.

[24]  Hai Zhao,et al.  SJTU at MRP 2019: A Transition-Based Multi-Task Parser for Cross-Framework Meaning Representation Parsing , 2019, CoNLL Shared Task.

[25]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[26]  Regina Barzilay,et al.  Hierarchical Low-Rank Tensors for Multilingual Transfer Parsing , 2015, EMNLP.

[27]  Timothy Dozat,et al.  Simpler but More Accurate Semantic Dependency Parsing , 2018, ACL.

[28]  Yijia Liu,et al.  HIT-SCIR at MRP 2019: A Unified Pipeline for Meaning Representation Parsing via Efficient Training and Effective Encoding , 2019, CoNLL.

[29]  Regina Barzilay,et al.  Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing , 2019, NAACL.

[30]  Jungo Kasai,et al.  Low-Resource Parsing with Crosslingual Contextualized Representations , 2019, CoNLL.

[31]  Noah A. Smith,et al.  A Simple, Fast, and Effective Reparameterization of IBM Model 2 , 2013, NAACL.

[32]  Kewei Tu,et al.  Second-Order Semantic Dependency Parsing with End-to-End Neural Networks , 2019, ACL.

[33]  Rudolf Rosa,et al.  CUNI x-ling: Parsing Under-Resourced Languages in CoNLL 2018 UD Shared Task , 2018, CoNLL Shared Task.