Linguistically inspired morphological inflection with a sequence to sequence model

Inflection is an essential part of every human language's morphology, yet little effort has been made to unify linguistic theory and computational methods in recent years. Methods of string manipulation are used to infer inflectional changes; our research question is whether a neural network would be capable of learning inflectional morphemes for inflection production in a similar way to a human in early stages of language acquisition. We are using an inflectional corpus (Metheniti and Neumann, 2020) and a single layer seq2seq model to test this hypothesis, in which the inflectional affixes are learned and predicted as a block and the word stem is modelled as a character sequence to account for infixation. Our character-morpheme-based model creates inflection by predicting the stem character-to-character and the inflectional affixes as character blocks. We conducted three experiments on creating an inflected form of a word given the lemma and a set of input and target features, comparing our architecture to a mainstream character-based model with the same hyperparameters, training and test sets. Overall for 17 languages, we noticed small improvements on inflecting known lemmas (+0.68%) but steadily better performance of our model in predicting inflected forms of unknown words (+3.7%) and small improvements on predicting in a low-resource scenario (+1.09%)

[1]  Barbara Landau,et al.  Function Morphemes in Young Children's Speech Perception and Production , 1990 .

[2]  P. Jusczyk,et al.  Sensitivity to discontinuous dependencies in language learners: evidence for limitations in processing space , 1998, Cognition.

[3]  J. Werker,et al.  Newborn infants’ sensitivity to perceptual cues to lexical and grammatical words , 1999, Cognition.

[4]  Erhard W. Hinrichs,et al.  The Tüba-D/Z Treebank: Annotating German with a Context-Free Backbone , 2004, LREC.

[5]  J. Weissenborn,et al.  The Recognition of Discontinuous Verbal Dependencies by German 19-Month-Olds: Evidence for Lexical and Structural Influences on Children's Early Processing Capacities , 2006 .

[6]  Josef van Genabith,et al.  Learning Morphology with Morfette , 2008, LREC.

[7]  Diane Poulin-Dubois,et al.  Early verb learning in 20-month-old Japanese-speaking children. , 2011, Journal of child language.

[8]  W. Dressler,et al.  On the role of morphological richness in the early development of noun and verb inflection , 2011 .

[9]  Rushen Shi,et al.  Initial morphological learning in preverbal infants , 2012, Cognition.

[10]  Toben H. Mintz.  The Segmentation of Sub-Lexical Morphemes in English-Learning 15-Month-Olds , 2013, Front. Psychology.

[11]  John DeNero,et al.  Supervised Learning of Complete Morphological Paradigms , 2013, NAACL.

[12]  Markus Forsberg,et al.  Semi-supervised learning of morphological paradigms and lexicons , 2014, EACL.

[13]  Markus Forsberg,et al.  Paradigm classification in supervised learning of morphology , 2015, HLT-NAACL.

[14]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[15]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[16]  Katharina Kann,et al.  Single-Model Encoder-Decoder with Explicit Morphological Representation for Reinflection , 2016, ACL.

[17]  Yonatan Belinkov,et al.  Improving Sequence to Sequence Learning for Morphological Inflection Generation: The BIU-MIT Systems for the SIGMORPHON 2016 Shared Task for Morphological Reinflection , 2016, SIGMORPHON.

[18]  Christo Kirov,et al.  Very-large Scale Parsing and Normalization of Wiktionary Morphological Paradigms , 2016, LREC.

[19]  Josef van Genabith,et al.  Neural Morphological Tagging from Characters for Morphologically Rich Languages , 2016, ArXiv.

[20]  Simon Clematide,et al.  Align and Copy: UZH at SIGMORPHON 2017 Shared Task for Morphological Reinflection , 2017, CoNLL.

[21]  Utpal Garain,et al.  Context Sensitive Lemmatization Using Two Successive Bidirectional Gated Recurrent Networks , 2017, ACL.

[22]  Yoav Goldberg,et al.  Morphological Inflection Generation with Hard Monotonic Attention , 2016, ACL.

[23]  Erhard W. Hinrichs,et al.  Seq2Seq or Perceptrons for Robust Lemmatization. An Empirical Examination , 2018 .

[24]  Simon Clematide,et al.  UZH at CoNLL-SIGMORPHON 2018 Shared Task on Universal Morphological Reinflection , 2018, CoNLL Shared Task.

[25]  Judit Ács.  BME-HAS System for CoNLL-SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection , 2018, CoNLL Shared Task.

[26]  Eleni Metheniti,et al.  Wikinflection: Massive Semi-Supervised Generation of Multilingual Inflectional Corpus from Wiktionary , 2018 .

[27]  Robert Weißgraeber,et al.  AX Semantics' Submission to the CoNLL-SIGMORPHON 2018 Shared Task , 2018, CoNLL Shared Task.

[28]  Ryan Cotterell,et al.  The CoNLL–SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection , 2018, CoNLL.

[29]  Sharon Goldwater,et al.  Context Sensitive Neural Lemmatization with Lematus , 2018, NAACL-HLT.

[30]  Sharon Goldwater,et al.  Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text , 2019, NAACL-HLT.

[31]  Ryan Cotterell,et al.  UniMorph 3.0: Universal Morphology , 2018, LREC.

[32]  Eleni Metheniti,et al.  Wikinflection Corpus: A (Better) Multilingual, Morpheme-Annotated Inflectional Corpus , 2020, LREC.