Cross-lingual Adaptation Using Universal Dependencies

We describe a cross-lingual adaptation method based on syntactic parse trees obtained from the Universal Dependencies (UD), which are consistent across languages, to develop classifiers in low-resource languages. The idea of UD parsing is to capture similarities as well as idiosyncrasies among typologically different languages. In this paper, we show that models trained using UD parse trees for complex NLP tasks can characterize very different languages. We study two tasks of paraphrase identification and semantic relation extraction as case studies. Based on UD parse trees, we develop several models using tree kernels and show that these models trained on the English dataset can correctly classify data of other languages e.g. French, Farsi, and Arabic. The proposed approach opens up avenues for exploiting UD parsing in solving similar cross-lingual tasks, which is very useful for languages that no labeled data is available for them.

[1]  Joakim Nivre.  Universal Dependencies: A Cross-Linguistic Perspective on Grammar and Lexicon , 2016 .

[2]  Roberto Basili,et al.  KeLP: a Kernel-based Learning Platform for Natural Language Processing , 2015, ACL.

[3]  George D. C. Cavalcanti,et al.  Combining sentence similarities measures to identify paraphrases , 2018, Comput. Speech Lang..

[4]  Nitin Madnani,et al.  Re-examining Machine Translation Metrics for Paraphrase Identification , 2012, NAACL.

[5]  Zhiyuan Liu,et al.  Relation Classification via Multi-Level Attention CNNs , 2016, ACL.

[6]  Alessandro Moschitti,et al.  Convolution Kernels on Constituent, Dependency and Sequential Structures for Relation Extraction , 2009, EMNLP.

[7]  Matteo Pagliardini,et al.  Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features , 2017, NAACL.

[8]  Guoyin Wang,et al.  Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms , 2018, ACL.

[9]  Sylvain Kahane,et al.  Multi-word annotation in syntactic treebanks - Propositions for Universal Dependencies , 2018, TLT.

[10]  Wiem Lahbib,et al.  A Hybrid Approach for Arabic Semantic Relation Extraction , 2013, FLAIRS Conference.

[11]  Philipp Koehn,et al.  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2016 .

[12]  Jacob Eisenstein,et al.  Discriminative Improvements to Distributional Sentence Similarity , 2013, EMNLP.

[13]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[14]  Christopher D. Manning,et al.  Enhanced English Universal Dependencies: An Improved Representation for Natural Language Understanding Tasks , 2016, LREC.

[15]  Jörg Tiedemann,et al.  Cross-Lingual Dependency Parsing with Universal Dependencies and Predicted PoS Labels , 2015, DepLing.

[16]  Zhiguo Wang,et al.  Sentence Similarity Learning by Lexical Decomposition and Composition , 2016, COLING.

[17]  Jan Hajic,et al.  UDPipe: Trainable Pipeline for Processing CoNLL-U Files Performing Tokenization, Morphological Analysis, POS Tagging and Parsing , 2016, LREC.

[18]  Saroj Kaushik,et al.  A Paraphrase and Semantic Similarity Detection System for User Generated Short-Text Content on Microblogs , 2016, COLING.

[19]  Jie Zhou,et al.  Paraphrase Identification Based on Weighted URAE, Unit Similarity and Context Correlation Feature , 2018, NLPCC.

[20]  Eneko Agirre,et al.  SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation , 2017, *SEMEVAL.

[21]  Heri Ramampiaro,et al.  A Deep Network Model for Paraphrase Detection in Short Text Messages , 2017, Inf. Process. Manag..

[22]  Preslav Nakov,et al.  SemEval-2007 Task 04: Classification of Semantic Relations between Nominals , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[23]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[24]  Hamidreza Chitsaz,et al.  SNPPhenA: a corpus for extracting ranked associations of single-nucleotide polymorphisms and phenotypes from literature , 2017, Journal of Biomedical Semantics.

[25]  Mark A. Przybocki,et al.  The Automatic Content Extraction (ACE) Program – Tasks, Data, and Evaluation , 2004, LREC.

[26]  Jun Guo,et al.  Designing an adaptive attention mechanism for relation classification , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[27]  Joakim Nivre,et al.  Universal Dependency Evaluation , 2017, UDW@NoDaLiDa.

[28]  Heshaam Faili,et al.  Toward a Multi-Representation Persian Treebank , 2018, 2018 9th International Symposium on Telecommunications (IST).

[29]  Nigel Collier,et al.  A Richer-but-Smarter Shortest Dependency Path with Attentive Augmentation for Relation Extraction , 2019, NAACL-HLT.

[30]  Jun Guo,et al.  An empirical convolutional neural network approach for semantic relation classification , 2016, Neurocomputing.

[31]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[32]  Pollet Samvelian,et al.  Introducing PersPred, a Syntactic and Semantic Database for Persian Complex Predicates , 2013, MWE@NAACL-HLT.

[33]  Guillaume Lample,et al.  Massively Multilingual Word Embeddings , 2016, ArXiv.

[34]  Chris Quirk,et al.  Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources , 2004, COLING.

[35]  Heshaam Faili,et al.  NSURL-2019 Task 7: Named Entity Recognition (NER) in Farsi , 2020, ArXiv.

[36]  Martin Potthast,et al.  CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies , 2018, CoNLL.

[37]  Jan Hajic,et al.  Parsing Universal Dependency Treebanks using Neural Networks and Search-Based Oracle Milan , 2016 .

[38]  M. de Rijke,et al.  Short Text Similarity with Word Embeddings , 2015, CIKM.

[39]  Christopher D. Manning,et al.  The Stanford Typed Dependencies Representation , 2008, CF+CDPE@COLING.

[40]  Young-Bum Kim,et al.  Cross-Lingual Transfer Learning for POS Tagging without Cross-Lingual Resources , 2017, EMNLP.

[41]  Slav Petrov,et al.  A Universal Part-of-Speech Tagset , 2011, LREC.

[42]  Guillaume Lample,et al.  Word Translation Without Parallel Data , 2017, ICLR.

[43]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[44]  Preslav Nakov,et al.  SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals , 2009, SEW@NAACL-HLT.

[45]  Robert Dale,et al.  Handbook of Natural Language Processing , 2001, Computational Linguistics.

[46]  Roberto Basili,et al.  Structured Lexical Similarity via Convolution Kernels on Dependency Trees , 2011, EMNLP.

[47]  Wei Shi,et al.  Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification , 2016, ACL.

[48]  Dongyan Zhao,et al.  Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling , 2015, EMNLP.

[49]  TiedemannJörg,et al.  Synthetic treebanking for cross-lingual dependency parsing , 2016 .

[50]  Clare R. Voss,et al.  Cross-lingual Structure Transfer for Relation and Event Extraction , 2019, EMNLP.

[51]  Alessandro Moschitti,et al.  Structural Representations for Learning Relations between Pairs of Texts , 2015, ACL.

[52]  Alessandro Moschitti,et al.  Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees , 2006, ECML.

[53]  Hai Zhao,et al.  Joint Learning of POS and Dependencies for Multilingual Universal Dependency Parsing , 2018, CoNLL.

[54]  Ngoc Thang Vu,et al.  Low-resource text classification using domain-adversarial learning , 2020, Comput. Speech Lang..

[55]  Heshaam Faili,et al.  Cross-Language Learning for Arabic Relation Extraction , 2018, ACLING.

[56]  Alessandro Moschitti,et al.  A Study on Convolution Kernels for Shallow Statistic Parsing , 2004, ACL.

[57]  Houfeng Wang,et al.  Bidirectional Recurrent Convolutional Neural Network for Relation Classification , 2016, ACL.

[58]  Yoshimasa Tsuruoka,et al.  Task-Oriented Learning of Word Embeddings for Semantic Relation Classification , 2015, CoNLL.

[59]  Hesham Faili,et al.  Automatic Wordnet Development for Low-Resource Languages using Cross-Lingual WSD , 2016, J. Artif. Intell. Res..

[60]  Joakim Nivre,et al.  Universal Dependency Annotation for Multilingual Parsing , 2013, ACL.

[61]  Jörg Tiedemann,et al.  Synthetic Treebanking for Cross-Lingual Dependency Parsing , 2016, J. Artif. Intell. Res..

[62]  Joakim Nivre,et al.  Universal Stanford dependencies: A cross-linguistic typology , 2014, LREC.

[63]  Kotagiri Ramamohanarao,et al.  Exploiting graph kernels for high performance biomedical relation extraction , 2018, Journal of Biomedical Semantics.

[64]  Alessandro Moschitti,et al.  State-of-the-Art Kernels for Natural Language Processing , 2012, ACL.

[65]  Suresh Manandhar,et al.  Dependency Based Embeddings for Sentence Classification Tasks , 2016, NAACL.

[66]  David Yarowsky,et al.  Cross-lingual Dependency Parsing Based on Distributed Representations , 2015, ACL.