Investigating the effects of i-complexity and e-complexity on the learnability of morphological systems

Research on cross-linguistic differences in morphological paradigms reveals a wide range of variation on many dimensions, including the number of categories expressed, the number of unique forms, and the number of inflectional classes. However, in an influential paper, Ackerman & Malouf (2013) argue that there is one dimension on which languages do not differ widely: in predictive structure. Predictive structure in a paradigm describes the extent to which forms predict each other, called i-complexity. Ackerman & Malouf (2013) show that although languages differ according to measure of surface paradigm complexity, called e-complexity, they tend to have low i-complexity. They conclude that morphological paradigms have evolved under a pressure for low i-complexity, such that even paradigms with very high e-complexity are relatively easy to learn so long as they have low i-complexity. While this would potentially explain why languages are able to maintain large paradigms, recent work by Johnson et al. (submitted) suggests that both neural networks and human learners may actually be more sensitive to e-complexity than i-complexity. Here we will build on this work, reporting a series of experiments under more realistic learning conditions which confirm that indeed, across a range of paradigms that vary in either e- or i-complexity, neural networks (LSTMs) are sensitive to both, but show a larger effect of e-complexity (and other measures associated with size and diversity of forms). In human learners, we fail to find any effect of i-complexity at all. Further, analysis of a large number of randomly generated paradigms show that e- and i-complexity are negatively correlated: paradigms with high e-complexity necessarily show low i-complexity.These findings suggest that the observations made by Ackerman & Malouf (2013) for natural language paradigms may stem from the nature of these measures rather than learning pressures specially attuned to i-complexity.

[1]  Kenny Smith,et al.  Predictive structure or paradigm size? Investigating the effects of i-complexity and e-complexity on the learnability of morphological systems , 2020 .

[2]  Kenny Smith,et al.  Assessing Integrative Complexity as a predictor of morphological learning using neural networks and artificial language learning , 2020 .

[3]  Roger Levy,et al.  Neural language models as psycholinguistic subjects: Representations of syntactic state , 2019, NAACL.

[4]  S. Kirby,et al.  Evolving artificial sign languages in the lab: From improvised gesture to systematic sign , 2018, Cognition.

[5]  Ryan Cotterell,et al.  On the Complexity and Typology of Inflectional Morphological Systems , 2018, TACL.

[6]  Edouard Grave,et al.  Colorless Green Recurrent Networks Dream Hierarchically , 2018, NAACL.

[7]  Per B. Brockhoff,et al.  lmerTest Package: Tests in Linear Mixed Effects Models , 2017 .

[8]  Robert Malouf,et al.  Abstractive morphological learning with a recurrent neural network , 2017 .

[9]  Emmanuel Dupoux,et al.  Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies , 2016, TACL.

[10]  Yang Xu,et al.  Historical Semantic Chaining and Efficient Communication: The Case of Container Names , 2016, Cogn. Sci..

[11]  Olivier Bonami,et al.  Joint predictiveness in inflectional paradigms , 2016 .

[12]  Simon Kirby,et al.  Simplicity and Specificity in Language: Domain-General Biases Have Domain-Specific Effects , 2016, Front. Psychol..

[13]  S. Kirby,et al.  Compression and communication in the cultural evolution of linguistic structure , 2015, Cognition.

[14]  E. Newport,et al.  Harmonic biases in child learners: In support of language universals , 2015, Cognition.

[15]  Anna L. Theakston,et al.  The ubiquity of frequency effects in first language acquisition , 2015, Journal of Child Language.

[16]  Terry Regier,et al.  Word Meanings across Languages Support Efficient Communication , 2015 .

[17]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[18]  Joshua F. Wiley,et al.  Growth Curve Analysis and Visualization Using R , 2014 .

[19]  Robert Malouf,et al.  Implicative organization facilitates morphological learning , 2014 .

[20]  Robert Malouf,et al.  Morphological Organization: The Low Conditional Entropy Conjecture , 2013 .

[21]  Elliott Moreton,et al.  Structure and Substance in Artificial-phonology Learning, Part I: Structure , 2012, Lang. Linguistics Compass.

[22]  E. Newport,et al.  Language learners restructure their input to facilitate efficient communication , 2012, Proceedings of the National Academy of Sciences.

[23]  P. Smolensky,et al.  Learning biases predict a word order universal , 2012, Cognition.

[24]  E. Newport,et al.  Getting it right by getting it wrong: When learners change languages , 2009, Cognitive Psychology.

[25]  Matthew Baerman,et al.  The Syntax-Morphology Interface: A Study of Syncretism , 2009 .

[26]  Morten H. Christiansen,et al.  Language as shaped by the brain. , 2008, The Behavioral and brain sciences.

[27]  Simon Kirby,et al.  Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language , 2008, Proceedings of the National Academy of Sciences.

[28]  James P. Blevins,et al.  Word-based morphology , 2006, Journal of Linguistics.

[29]  Ryan Keith Shosted,et al.  Correlating complexity: A typological approach , 2006 .

[30]  Carla L. Hudson Kam,et al.  Regularizing Unpredictable Variation: The Roles of Adult and Child Learners in Language Formation and Change , 2005 .

[31]  Donald Favareau.  The Symbolic Species: The Co-evolution of Language and the Brain , 1998 .

[32]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[33]  M. Aronoff.  Morphology by Itself: Stems and Inflectional Classes , 1993 .

[34]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[35]  Jennifer Culbertson,et al.  Something about "us": Learning first person pronoun systems , 2019, CogSci.

[36]  Robert Malouf.  The No Blur Principle Effects as an Emergent Property of Language Systems , 2015 .

[37]  Simon Kirby,et al.  Word Meanings Evolve to Selectively Preserve Distinctions on Salient Dimensions , 2015, Cogn. Sci..

[38]  Florian Lionnet,et al.  Logical Complexity in Morphological Learning : Effects of Structure and Null / Overt Affixation on Learning Paradigms , 2014 .

[39]  G. Corbett.  Suppletion: Typology, markedness, complexity , 2009 .

[40]  Robert Malouf,et al.  Parts and wholes : Patterns of relatedness in complex morphological systems and why they matter , 2007 .

[41]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[42]  Elissa L. Newport,et al.  Novelty and regularization : the effect of novel instances on rule formation , 2005 .

[43]  J. Elman.  Distributed representations, simple recurrent networks, and grammatical structure , 1991, Machine Learning.

[44]  Simon Kirby,et al.  Linguistic Evolution Through Language Acquisition: Learning, bottlenecks and the evolution of recursive syntax , 2002 .

[45]  Michael I. Jordan.  Serial Order: A Parallel Distributed Processing Approach , 1997 .

[46]  Materials and Methods Figs. S1 to S9 Tables S1 to S6 References , 2022 .