TRACX2: a connectionist autoencoder using graded chunks to model infant visual statistical learning

Even newborn infants are able to extract structure from a stream of sensory inputs; yet how this is achieved remains largely a mystery. We present a connectionist autoencoder model, TRACX2, that learns to extract sequence structure by gradually constructing chunks, storing these chunks in a distributed manner across its synaptic weights and recognizing these chunks when they re-occur in the input stream. Chunks are graded rather than all-or-nothing in nature. As chunks are learnt their component parts become more and more tightly bound together. TRACX2 successfully models the data from five experiments from the infant visual statistical learning literature, including tasks involving forward and backward transitional probabilities, low-salience embedded chunk items, part-sequences and illusory items. The model also captures performance differences across ages through the tuning of a single-learning rate parameter. These results suggest that infant statistical learning is underpinned by the same domain-general learning mechanism that operates in auditory statistical learning and, potentially, in adult artificial grammar learning. This article is part of the themed issue ‘New frontiers for statistical learning in the cognitive sciences’.

[1]  A. Vinter,et al.  PARSER: A Model for Word Segmentation , 1998 .

[2]  R. Baayen,et al.  Shifting paradigms: gradient structure in morphology , 2005, Trends in Cognitive Sciences.

[3]  D. Mareschal,et al.  Local redundancy governs infants' spontaneous orienting to visual-temporal sequences. , 2013, Child development.

[4]  Denis Mareschal,et al.  TRACX: a recognition-based connectionist framework for sequence segmentation and chunk extraction. , 2011, Psychological review.

[5]  James L. McClelland,et al.  Graded State Machines: The Representation of Temporal Contingencies in Simple Recurrent Networks , 2005, Machine Learning.

[6]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[7]  P. Perruchet,et al.  Implicit learning and statistical learning: one phenomenon, two approaches , 2006, Trends in Cognitive Sciences.

[8]  Dima Amso,et al.  Across space and time: infants learn from backward and forward visual statistics. , 2017, Developmental science.

[9]  Joanne Arciuli,et al.  The multi-component nature of statistical learning , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[10]  Paavo Alku,et al.  Statistical language learning in neonates revealed by event-related brain potentials , 2009, BMC Neuroscience.

[11]  T. A. Cartwright,et al.  Distributional regularity and phonotactic constraints are useful for segmentation , 1996, Cognition.

[12]  Robert M. French,et al.  TRACX 2.0: A memory-based, biologically-plausible model of sequence segmentation and chunk extraction , 2014, CogSci.

[13]  Scott P. Johnson,et al.  ARTICLE WITH PEER COMMENTARIES AND RESPONSE Learning to perceive object unity: a connectionist account , 2002 .

[14]  M G Pêcheux,et al.  Visual habituation in human infants: development and rearing circumstances , 1988, Psychological research.

[15]  A. Vinter,et al.  The self-organizing consciousness. , 2002, The Behavioral and brain sciences.

[16]  Denis Mareschal,et al.  Mechanisms of Categorization in Infancy. , 2000, Infancy : the official journal of the International Society on Infant Studies.

[17]  Richard N Aslin,et al.  Bayesian learning of visual chunks by human observers , 2008, Proceedings of the National Academy of Sciences.

[18]  Jessica F. Hay,et al.  Learning in reverse: Eight-month-old infants track backward transitional probabilities , 2009, Cognition.

[19]  J. Pine,et al.  Chunking mechanisms in human learning , 2001, Trends in Cognitive Sciences.

[20]  Erik D. Thiessen,et al.  The extraction and integration framework: a two-process account of statistical learning. , 2013, Psychological bulletin.

[21]  Mark S. Seidenberg,et al.  Graded semantic and phonological similarity effects in priming: evidence for a distributed connectionist approach to morphology. , 2007, Journal of experimental psychology. General.

[22]  E. W. Ames,et al.  A multifactor model of infant preferences for novel and familiar stimuli. , 1988 .

[23]  J. Colombo,et al.  Infant visual habituation , 2009, Neurobiology of Learning and Memory.

[24]  Arnaud Rey,et al.  Lexical and Sublexical Units in Speech Perception , 2009, Cogn. Sci..

[25]  Scott P. Johnson,et al.  Visual statistical learning in the newborn infant , 2011, Cognition.

[26]  Scott P. Johnson,et al.  Visual statistical learning in infancy: evidence for a domain general learning mechanism , 2002, Cognition.

[27]  L. Frank The Society for Research in Child Development , 1935 .

[28]  Jacques Mehler,et al.  The surprising power of statistical learning: When fragment knowledge leads to false memories of unheard words , 2009 .

[29]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[30]  Pierre Perruchet,et al.  A role for backward transitional probabilities in word segmentation? , 2008, Memory & cognition.

[31]  Gert Westermann,et al.  From perceptual to language-mediated categorization , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[32]  Scott P. Johnson,et al.  When learning goes beyond statistics: Infants represent visual sequences in terms of chunks , 2018, Cognition.

[33]  B. Roder,et al.  Infants' Preferences for Familiarity and Novelty During the Course of Visual Processing. , 2000, Infancy : the official journal of the International Society on Infant Studies.

[34]  R. French,et al.  A connectionist account of asymmetric category learning in early infancy. , 2000, Developmental psychology.

[35]  D. Mareschal,et al.  Modeling Infant Speech Sound Discrimination Using Simple Associative Networks. , 2001, Infancy : the official journal of the International Society on Infant Studies.

[36]  A. Newell Unified Theories of Cognition , 1990 .

[37]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[38]  E. Newport,et al.  Computation of Conditional Probability Statistics by 8-Month-Old Infants , 1998 .

[39]  Martial Mermillod,et al.  The role of bottom-up processing in perceptual categorization by 3- to 4-month-old infants: simulations and data. , 2004, Journal of experimental psychology. General.

[40]  Robert M. French,et al.  Asymmetric interference in 3- to 4-month-olds' sequential category learning , 2002, Cogn. Sci..

[41]  Scott P. Johnson,et al.  Statistical Learning Across Development: Flexible Yet Constrained , 2013, Front. Psychology.

[42]  Philip I. Pavlik,et al.  iMinerva: A Mathematical Model of Distributional Statistical Learning , 2013, Cogn. Sci..

[43]  Stuart Marcovitch,et al.  Sequence learning in infancy: the independent contributions of conditional probability and pair frequency information. , 2009, Developmental science.

[44]  Denis Mareschal,et al.  An Interacting Systems Model of Infant Habituation , 2004, Journal of Cognitive Neuroscience.

[45]  Jordan B. Pollack,et al.  Implications of Recursive Distributed Representations , 1988, NIPS.

[46]  Erik D. Thiessen,et al.  What's statistical about learning? Insights from modelling statistical learning as a set of memory processes , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[47]  E. Pothos Theories of artificial grammar learning. , 2007, Psychological bulletin.

[48]  A. Reber Implicit learning of artificial grammars , 1967 .