Pushing the Limits of Low-Resource Morphological Inflection

Recent years have seen exceptional strides in the task of automatic morphological inflection generation. However, for a long tail of languages the necessary resources are hard to come by, and state-of-the-art neural methods that work well under higher resource settings perform poorly in the face of a paucity of data. In response, we propose a battery of improvements that greatly improve performance under such low-resource conditions. First, we present a novel two-step attention architecture for the inflection decoder. In addition, we investigate the effects of cross-lingual transfer from single and multiple languages, as well as monolingual data hallucination. The macro-averaged accuracy of our models outperforms the state-of-the-art by 15 percentage points. Also, we identify the crucial factors for success with cross-lingual transfer for morphological inflection: typological similarity and a common representation across languages.

[1]  Graham Neubig,et al.  Controllable Invariance through Adversarial Feature Learning , 2017, NIPS.

[2]  Yoav Goldberg,et al.  Morphological Inflection Generation with Hard Monotonic Attention , 2016, ACL.

[3]  Christopher D. Manning,et al.  Stanford Neural Machine Translation Systems for Spoken Language Domains , 2015, IWSLT.

[4]  Guillaume Lample,et al.  Unsupervised Machine Translation Using Monolingual Corpora Only , 2017, ICLR.

[5]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[6]  Ryan Cotterell,et al.  The CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection , 2018, CoNLL.

[7]  Ryan Cotterell,et al.  The SIGMORPHON 2016 Shared Task—Morphological Reinflection , 2016, SIGMORPHON.

[8]  Ryan Cotterell,et al.  CoNLL-SIGMORPHON 2017 Shared Task: Universal Morphological Reinflection in 52 Languages , 2017, CoNLL.

[9]  Graham Neubig,et al.  Morphological Inflection Generation with Multi-space Variational Encoder-Decoders , 2017, CoNLL.

[10]  Lane Schwartz,et al.  Bootstrapping a Neural Morphological Analyzer for St. Lawrence Island Yupik from a Finite-State Transducer , 2019 .

[11]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[12]  Kevin Duh,et al.  DyNet: The Dynamic Neural Network Toolkit , 2017, ArXiv.

[13]  Kevin Knight,et al.  Multi-Source Neural Translation , 2016, NAACL.

[14]  Michael C. Ewing,et al.  Indonesian: A Comprehensive Grammar , 2010 .

[15]  Ryan Cotterell,et al.  One-Shot Neural Cross-Lingual Transfer for Paradigm Completion , 2017, ACL.

[16]  André F. T. Martins,et al.  IT–IST at the SIGMORPHON 2019 Shared Task: Sparse Two-headed Models for Inflection , 2019, Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology.

[17]  Graham Neubig,et al.  Rapid Adaptation of Neural Machine Translation to New Languages , 2018, EMNLP.

[18]  David Chiang,et al.  Leveraging translations for speech transcription in low-resource settings , 2018, INTERSPEECH.

[19]  Judit Ács.  BME-HAS System for CoNLL-SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection , 2018, CoNLL Shared Task.

[20]  Ryan Cotterell,et al.  The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection , 2019, Proceedings of the 16th Workshop on Computational Research in Phonetics, Phonology, and Morphology.

[21]  Gholamreza Haffari,et al.  Incorporating Structural Alignment Biases into an Attentional Neural Translation Model , 2016, NAACL.

[22]  Claire Cardie,et al.  Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification , 2016, TACL.

[23]  Mark Aronoff,et al.  3. The verbal morphology of Maltese , 2003 .

[24]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[25]  Ryan Cotterell,et al.  Exact Hard Monotonic Attention for Character-Level Transduction , 2019, ACL.

[26]  Patrick Littell,et al.  URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors , 2017, EACL.

[27]  Ramón Fernández Astudillo,et al.  From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification , 2016, ICML.

[28]  Ling Liu,et al.  Data Augmentation for Morphological Reinflection , 2017, CoNLL.

[29]  Samy Bengio,et al.  Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks , 2015, NIPS.

[30]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[31]  Katharina Kann,et al.  Training Data Augmentation for Low-Resource Morphological Inflection , 2017, CoNLL.

[32]  Daan van Esch,et al.  Automatic Keyboard Layout Design for Low-Resource Latin-Script Languages , 2019, ArXiv.

[33]  Scott Heath,et al.  Building Speech Recognition Systems for Language Documentation: The CoEDL Endangered Language Pipeline and Inference System (ELPIS) , 2018, SLTU.

[34]  Yonatan Belinkov,et al.  Improving Sequence to Sequence Learning for Morphological Inflection Generation: The BIU-MIT Systems for the SIGMORPHON 2016 Shared Task for Morphological Reinflection , 2016, SIGMORPHON.

[35]  Ryan Cotterell,et al.  UniMorph 3.0: Universal Morphology , 2018, LREC.