PD3: Better Low-Resource Cross-Lingual Transfer By Combining Direct Transfer and Annotation Projection

We consider unsupervised cross-lingual transfer on two tasks, viz., sentence-level argumentation mining and standard POS tagging. We combine direct transfer using bilingual embeddings with annotation projection, which projects labels across unlabeled parallel data. We do so by either merging respective source and target language datasets or alternatively by using multi-task learning. Our combination strategy considerably improves upon both direct transfer and projection with few available parallel sentences, the most realistic scenario for many low-resource target languages.

[1]  Ruslan Salakhutdinov,et al.  Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks , 2016, ICLR.

[2]  Oscar Täckström.  Nudging the Envelope of Direct Transfer Methods for Multilingual Named Entity Recognition , 2012, HLT-NAACL 2012.

[3]  Eneko Agirre,et al.  Unsupervised Neural Machine Translation , 2017, ICLR.

[4]  Slav Petrov,et al.  Multi-Source Transfer of Delexicalized Dependency Parsers , 2011, EMNLP.

[5]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[6]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[7]  Iryna Gurevych,et al.  Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need! , 2018, COLING.

[8]  Ahmet Aker,et al.  Projection of Argumentative Corpora from Source to Target Languages , 2017, ArgMining@EMNLP.

[9]  Barbara Plank,et al.  Multilingual Projection for Parsing Truly Low-Resource Languages , 2016, TACL.

[10]  Joakim Nivre,et al.  Token and Type Constraints for Cross-Lingual Part-of-Speech Tagging , 2013, TACL.

[11]  Joakim Nivre,et al.  Target Language Adaptation of Discriminative Transfer Parsers , 2013, NAACL.

[12]  Iryna Gurevych,et al.  Multi-Task Learning for Argumentation Mining in Low-Resource Settings , 2018, NAACL.

[13]  Guillaume Lample,et al.  Unsupervised Machine Translation Using Monolingual Corpora Only , 2017, ICLR.

[14]  Sampo Pyysalo,et al.  Universal Dependencies v1: A Multilingual Treebank Collection , 2016, LREC.

[15]  Marcin Junczys-Dowmunt,et al.  The United Nations Parallel Corpus v1.0 , 2016, LREC.

[16]  Iryna Gurevych,et al.  ArgumenText: Searching for Arguments in Heterogeneous Sources , 2018, NAACL.

[17]  David Yarowsky,et al.  Inducing Multilingual Text Analysis Tools via Robust Projection across Aligned Corpora , 2001, HLT.

[18]  Eneko Agirre,et al.  Learning bilingual word embeddings with (almost) no bilingual data , 2017, ACL.

[19]  Anders Søgaard,et al.  Improving historical spelling normalization with bi-directional LSTMs and multi-task learning , 2016, COLING.

[20]  Iryna Gurevych,et al.  Parsing Argumentation Structures in Persuasive Essays , 2016, CL.

[21]  Manfred Stede,et al.  From Argument Diagrams to Argumentation Mining in Texts: A Survey , 2013, Int. J. Cogn. Informatics Nat. Intell..

[22]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[23]  Mark Steedman,et al.  A massively parallel corpus: the Bible in 100 languages , 2014, Lang. Resour. Evaluation.

[24]  Ryan Cotterell,et al.  Cross-lingual Character-Level Neural Morphological Tagging , 2017, EMNLP.

[25]  Marie-Francine Moens,et al.  Bilingual Word Embeddings from Non-Parallel Document-Aligned Data Applied to Bilingual Lexicon Induction , 2015, ACL.

[26]  Claire Cardie,et al.  Argument Mining with Structured SVMs and RNNs , 2017, ACL.

[27]  Noah A. Smith,et al.  A Simple, Fast, and Effective Reparameterization of IBM Model 2 , 2013, NAACL.

[28]  Xuanjing Huang,et al.  Adversarial Multi-task Learning for Text Classification , 2017, ACL.

[29]  Joachim Bingel,et al.  Sluice networks: Learning what to share between loosely related tasks , 2017, ArXiv.

[30]  Iryna Gurevych,et al.  Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations , 2018, 1803.01400.

[31]  Marie-Francine Moens,et al.  Argumentation mining: the detection, classification and structure of arguments in text , 2009, ICAIL.

[32]  Regina Barzilay,et al.  Ten Pairs to Tag – Multilingual POS Tagging via Coarse Mapping between Embeddings , 2016, NAACL.

[33]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[34]  Marie-Francine Moens,et al.  Argumentation mining: How can a machine acquire common sense and world knowledge? , 2018, Argument Comput..

[35]  Vincent Ng,et al.  End-to-End Argumentation Mining in Student Essays , 2016, NAACL.

[36]  Slav Petrov,et al.  Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections , 2011, ACL.

[37]  Anders Søgaard,et al.  Deep multi-task learning with low level tasks supervised at lower layers , 2016, ACL.

[38]  Iryna Gurevych,et al.  Annotating Argument Components and Relations in Persuasive Essays , 2014, COLING.