Early lexical development in a self-organizing neural network

In this paper we present a self-organizing neural network model of early lexical development called DevLex. The network consists of two self-organizing maps (a growing semantic map and a growing phonological map) that are connected via associative links trained by Hebbian learning. The model captures a number of important phenomena that occur in early lexical acquisition by children, as it allows for the representation of a dynamically changing linguistic environment in language learning. In our simulations, DevLex develops topographically organized representations for linguistic categories over time, models lexical confusion as a function of word density and semantic similarity, and shows age-of-acquisition effects in the course of learning a growing lexicon. These results match up with patterns from empirical research on lexical development, and have significant implications for models of language acquisition based on self-organizing neural networks.

[1]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[2]  Risto Miikkulainen,et al.  Subsymbolic natural language processing - an integrated model of scripts, lexicon, and memory , 1993, Neural network modeling and connectionism.

[3]  Ping Li,et al.  The Acquisition of Lexical and Grammatical Aspect in Self-Organizing Feature-Map Model , 2000 .

[4]  Catriona M. Morrison,et al.  Real age-of-acquisition effects in lexical retrieval. , 1998, Journal of experimental psychology. Learning, memory, and cognition.

[5]  B. MacWhinney Models of the emergence of language. , 1998, Annual review of psychology.

[6]  Igor Farkas,et al.  Modeling the Development of Lexicon with a Growing Self-Organizing Map , 2002, JCIS.

[7]  M. Raijmakers Rethinking innateness: A connectionist perspective on development. , 1997 .

[8]  Eric Wanner,et al.  Language acquisition: the state of the art , 1982 .

[9]  P. Bloom How children learn the meanings of words , 2000 .

[10]  Peter Broeder,et al.  Models of Language Acquisition: Inductive and Deductive Approaches , 2001 .

[11]  Ping Li,et al.  Competition : A Connectionist Model of the Learning of English Reversive Prefixes , 2010 .

[12]  T. Shultz Computational Developmental Psychology , 2003 .

[13]  T. Sejnowski,et al.  Irresistible environment meets immovable neurons , 1997, Behavioral and Brain Sciences.

[14]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[15]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[16]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[17]  P. Ladefoged A course in phonetics , 1975 .

[18]  Chris Sinha,et al.  Symbol Grounding or the Emergence of Symbols? Vocabulary Growth in Children and a Connectionist Net , 1992 .

[19]  J. Reznick,et al.  Developmental and stylistic variation in the composition of early vocabulary , 1994, Journal of Child Language.

[20]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[21]  Garrison W. Cottrell,et al.  The Early Word Catches the Weights , 2000, NIPS.

[22]  A. Caramazza,et al.  Lexical organization of nouns and verbs in the brain , 1991, Nature.

[23]  I. Farkas,et al.  DevLex: a self-organizing neural network model of the development of lexicon , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[24]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[25]  L. R Gleitman,et al.  Proceedings of the twenty-second annual conference of the cognitive science society , 2000 .

[26]  Stephen Grossberg,et al.  Art 2: Self-Organization Of Stable Category Recognition Codes For Analog Input Patterns , 1988, Other Conferences.

[27]  David G. Stork,et al.  Pattern Classification , 1973 .

[28]  E. Dromi Early Lexical Development , 1987 .

[29]  Thorsten Piske,et al.  An Integrated View of Language Development , 2003, English and American Studies in German.

[30]  J. Altarriba,et al.  A Self-Organizing Connectionist Model of Bilingual Processing , 2001 .

[31]  Ping Li,et al.  PatPho: A phonological pattern generator for neural networks , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[32]  E. Gould,et al.  Neurogenesis in adulthood: a possible role in learning , 1999, Trends in Cognitive Sciences.

[33]  John Lyons,et al.  语义学引论 = Linguistic Semantics , 2000 .

[34]  Ping Li,et al.  A self-organizing neural network model of the acquisition of word meaning , 2001 .

[35]  B. MacWhinney The CHILDES project: tools for analyzing talk , 1992 .

[36]  Andrew W. Ellis,et al.  ROLES OF WORD FREQUENCY AND AGE OF ACQUISITION IN WORD NAMING AND LEXICAL DECISION , 1995 .

[37]  S. Pinker,et al.  The Language Instinct: How the Mind Creates Language , 1994 .

[38]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[39]  Risto Miikkulainen,et al.  Visualizing High-Dimensional Structure with the Incremental Grid Growing Neural Network , 1995, ICML.

[40]  S. Pinker The language instinct : how the mind creates language , 1995 .

[41]  William D. Marslen-Wilson,et al.  Universals in Morphological Representation: Evidence from Italian , 1997 .

[42]  Richard D. Hichwa,et al.  A neural basis for lexical retrieval , 1996, Nature.

[43]  Stephen R. Marsland,et al.  A self-organising network that grows when required , 2002, Neural Networks.

[44]  Melissa Bowerman,et al.  Systematizing semantic knowledge: Changes over time in the child's organization of word meaning , 1978 .

[45]  J. Fletcher,et al.  The changing nervous system : neurobehavioral consequences of early brain disorders , 1999 .

[46]  R. French Catastrophic Forgetting in Connectionist Networks , 2006 .

[47]  S. Stoness Generalization , Representation , and Recovery in a Self-Organizing Feature-Map Model of Language Acquisition , 1999 .

[48]  Melissa Bowerman,et al.  Reorganizational processes in lexical and syntactic development , 1982 .

[49]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[50]  Thomas R. Shultz,et al.  Connectionist Models of Development: Developmental Processes in Real and Artificial Neural Networks , 2003 .

[51]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[52]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .

[53]  Ping Li Language Acquisition in a Self-Organising Neural Network Model , 2004 .

[54]  Jeanette Altarriba,et al.  Bilingual sentence processing , 2002 .

[55]  Ping Li,et al.  3 A self-organizing connectionist model of bilingual processing , 2002 .

[56]  Friedemann Pulverm Uuml,et al.  Words in the brain's language , 1999 .

[57]  Dedre Gentner,et al.  Why Nouns Are Learned before Verbs: Linguistic Relativity Versus Natural Partitioning. Technical Report No. 257. , 1982 .

[58]  C. Fellbaum An Electronic Lexical Database , 1998 .

[59]  E. Bates,et al.  New Directions in Research on Language Development , 1993 .

[60]  M. Tomasello,et al.  Variability in early communicative development. , 1994, Monographs of the Society for Research in Child Development.

[61]  Mark H. Johnson Brain development and cognition : a reader , 1993 .

[62]  R. Miikkulainen Dyslexic and Category-Specific Aphasic Impairments in a Self-Organizing Feature Map Model of the Lexicon , 1997, Brain and Language.

[63]  T. Kohonen,et al.  Self-organizing semantic maps , 1989, Biological Cybernetics.

[64]  Michael Gasser,et al.  Learning Nouns and Adjectives: A Connectionist Account , 1998 .

[65]  Ping Li,et al.  The Acquisition of Word Meaning through Global Lexical Co-occurrences , 2000 .

[66]  P. Luce,et al.  Similarity neighbourhoods of words in young children's lexicons , 1990, Journal of Child Language.

[67]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[68]  Linda B. Smith,et al.  A Curvilinear Trend in Naming Errors as a Function of Early Vocabulary Growth , 1997, Cognitive Psychology.

[69]  S. Stoness,et al.  Proceedings of the Twenty First Annual Conference of the Cognitive Science Society , 1999 .

[70]  Noam Chomsky Reflections on Language. , 1977 .

[71]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[72]  James W. Deutsch The Mind within the Net: Models of Learning, Thinking and Acting , 2001 .

[73]  J. Reznick,et al.  Early lexical acquisition: rate, content, and the vocabulary spurt , 1990, Journal of Child Language.

[74]  L. Fenson,et al.  Lexical development norms for young children , 1996 .

[75]  T. Kohonen Self-organized formation of topographically correct feature maps , 1982 .

[76]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[77]  Ping Li,et al.  Neural representations of nouns and verbs in Chinese: an fMRI study , 2004, NeuroImage.

[78]  J. Elman An alternative view of the mental lexicon , 2004, Trends in Cognitive Sciences.

[79]  James L. McClelland,et al.  On learning the past-tenses of English verbs: implicit rules or parallel distributed processing , 1986 .

[80]  M. Harm Building Large Scale Distributed Semantic Feature Sets with WordNet , 2022 .

[81]  J. Elman Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[82]  M. L. Lambon Ralph,et al.  Age of acquisition effects in adult lexical processing reflect loss of plasticity in maturing systems: insights from connectionist networks. , 2000, Journal of experimental psychology. Learning, memory, and cognition.