Abstraction and generalization in statistical learning: implications for the relationship between semantic types and episodic tokens

Statistical approaches to emergent knowledge have tended to focus on the process by which experience of individual episodes accumulates into generalizable experience across episodes. However, there is a seemingly opposite, but equally critical, process that such experience affords: the process by which, from a space of types (e.g. onions—a semantic class that develops through exposure to individual episodes involving individual onions), we can perceive or create, on-the-fly, a specific token (a specific onion, perhaps one that is chopped) in the absence of any prior perceptual experience with that specific token. This article reviews a selection of statistical learning studies that lead to the speculation that this process—the generation, on the basis of semantic memory, of a novel episodic representation—is itself an instance of a statistical, in fact associative, process. The article concludes that the same processes that enable statistical abstraction across individual episodes to form semantic memories also enable the generation, from those semantic memories, of representations that correspond to individual tokens, and of novel episodic facts about those tokens. Statistical learning is a window onto these deeper processes that underpin cognition. This article is part of the themed issue ‘New frontiers for statistical learning in the cognitive sciences’.

[1]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[2]  Z. Dienes,et al.  Modality Independence of Implicitly Learned Grammatical Knowledge , 2004 .

[3]  R. Henson,et al.  How schema and novelty augment memory formation , 2012, Trends in Neurosciences.

[4]  G. Winocur,et al.  Episodic Memory and Beyond: The Hippocampus and Neocortex in Transformation. , 2016, Annual review of psychology.

[5]  Jon A. Willits,et al.  Models of Semantic Memory , 2015 .

[6]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[7]  Z. Dienes,et al.  On the modality independence of implicitly learned grammatical knowledge , 1995 .

[8]  S. Thompson-Schill,et al.  Putting concepts into context , 2016, Psychonomic bulletin & review.

[9]  Gerry Altmann,et al.  Mapping across Domains Without Feedback: A Neural Network Model of Transfer of Implicit Knowledge , 1999, Cogn. Sci..

[10]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[11]  R. Tunney,et al.  Two modes of transfer in artificial grammar learning. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[12]  Hallvard Røe Evensmoen,et al.  Long-axis specialization of the human hippocampus , 2013, Trends in Cognitive Sciences.

[13]  H. Eichenbaum,et al.  Memory, amnesia, and the hippocampal system , 1993 .

[14]  D. Hassabis,et al.  Deconstructing episodic memory with construction , 2007, Trends in Cognitive Sciences.

[15]  Pierre Perruchet,et al.  Synthetic grammar learning: Implicit rule abstraction or explicit fragmentary knowledge? Journal of , 1990 .

[16]  L. Tyler,et al.  Binding crossmodal object features in perirhinal cortex. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Axel Cleeremans,et al.  Computational Models of Implicit Learning , 2019, Implicit Learning.

[18]  A. Hupbach,et al.  Nap-dependent learning in infants. , 2009, Developmental science.

[19]  B. Wyble,et al.  The simultaneous type, serial token model of temporal attention and working memory. , 2007, Psychological review.

[20]  A. Vinter,et al.  PARSER: A Model for Word Segmentation , 1998 .

[21]  Marc W. Howard,et al.  Constructing Semantic Representations From a Gradually Changing Representation of Temporal Context , 2011, Top. Cogn. Sci..

[22]  A. Friederici The brain basis of language processing: from structure to function. , 2011, Physiological reviews.

[23]  David E. Warren,et al.  Hippocampal Amnesia Impairs All Manner of Relational Memory , 2008, Frontiers in human neuroscience.

[24]  Sharon L. Thompson-Schill,et al.  The Effect of Object State-Changes on Event Processing: Do Objects Compete with Themselves? , 2012, The Journal of Neuroscience.

[25]  Axel Cleeremans,et al.  Mechanisms of Implicit Learning: Connectionist Models of Sequence Processing , 1993 .

[26]  Lauren L. Long,et al.  Theta variation and spatiotemporal scaling along the septotemporal axis of the hippocampus , 2015, Front. Syst. Neurosci..

[27]  B. Scholl Object Persistence in Philosophy and Psychology , 2007 .

[28]  Irene E. Nagel,et al.  Human Aging Magnifies Genetic Effects on Executive Functioning and Working Memory , 2008, Frontiers in human neuroscience.

[29]  N. Cohen,et al.  Relational Memory and the Hippocampus: Representations and Methods , 2009, Front. Neurosci..

[30]  M. Botvinick,et al.  Statistical learning of temporal community structure in the hippocampus , 2016, Hippocampus.

[31]  P. Perruchet,et al.  Implicit learning and statistical learning: one phenomenon, two approaches , 2006, Trends in Cognitive Sciences.

[32]  E. Tulving,et al.  Episodic and semantic memory , 1972 .

[33]  Erik D. Thiessen,et al.  Statistical learning of language: Theory, validity, and predictions of a statistical learning account of language acquisition , 2015 .

[34]  Erik D. Thiessen,et al.  The extraction and integration framework: a two-process account of statistical learning. , 2013, Psychological bulletin.

[35]  Nicholas C. Hindy,et al.  A cortical network for the encoding of object change. , 2015, Cerebral cortex.

[36]  James L. McClelland,et al.  Sentence comprehension: A parallel distributed processing approach , 1989, Language and Cognitive Processes.

[37]  Jake Kurczek,et al.  Cohesion, coherence, and declarative memory: Discourse patterns in individuals with hippocampal amnesia , 2011, Aphasiology.

[38]  Sarah Brown-Schmidt,et al.  Hippocampal contributions to language: evidence of referential processing deficits in amnesia. , 2013, Journal of experimental psychology. General.

[39]  Demis Hassabis,et al.  The construction system of the brain , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[40]  K L Shapiro,et al.  Temporary suppression of visual processing in an RSVP task: an attentional blink? . , 1992, Journal of experimental psychology. Human perception and performance.

[41]  Brendan T. Johns,et al.  Generating structure from experience: A retrieval-based model of language processing. , 2015, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[42]  Curt Burgess,et al.  Producing high-dimensional semantic spaces from lexical co-occurrence , 1996 .

[43]  Douglas L. Hintzman,et al.  MINERVA 2: A simulation model of human memory , 1984 .

[44]  Rebecca L Gómez,et al.  Do infants retain the statistics of a statistical learning experience? Insights from a developmental cognitive neuroscience perspective , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[45]  Linda B. Smith,et al.  Infants rapidly learn word-referent mappings via cross-situational statistics , 2008, Cognition.

[46]  G. Altmann Learning and development in neural networks – the importance of prior experience , 2002, Cognition.

[47]  James L. McClelland,et al.  Generalization Through the Recurrent Interaction of Episodic Memories , 2012, Psychological review.

[48]  Gerry T. M. Altmann,et al.  The Ascent of Babel: An Exploration of Language, Mind, and Understanding , 1997 .

[49]  Rebecca L. Gómez,et al.  Naps Promote Abstraction in Language-Learning Infants , 2006, Psychological science.

[50]  James L. McClelland,et al.  What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated , 2016, Trends in Cognitive Sciences.

[51]  S. Pollmann,et al.  Retinotopic Activation in Response to Subjective Contours in Primary Visual Cortex , 2008, Frontiers in human neuroscience.

[52]  Mark F. St. John,et al.  The Story Gestalt: A Model of Knowledge-Intensive Processes in Text Comprehension , 1992, Cogn. Sci..

[53]  Michael N Jones,et al.  Representing word meaning and order information in a composite holographic lexicon. , 2007, Psychological review.

[54]  John R. Vokey,et al.  Abstract analogies and abstracted grammars: Comments on Reber (1989) and Mathews et al. (1989). , 1991 .

[55]  Z. Dienes,et al.  Rule learning by seven-month-old infants and neural networks , 1999 .

[56]  Jeffrey M. Zacks,et al.  A Computational Model of Event Segmentation From Perceptual Prediction , 2007, Cogn. Sci..

[57]  Nicholas B. Turk-Browne,et al.  Complementary learning systems within the hippocampus: A neural network modeling approach to reconciling episodic memory with statistical learning , 2016, bioRxiv.

[58]  Sharon L. Thompson-Schill,et al.  Competition between Mutually Exclusive Object States in Event Comprehension , 2015, Journal of Cognitive Neuroscience.

[59]  Sharon L. Thompson-Schill,et al.  Semantic Memory , 2019 .

[60]  Erik D. Thiessen,et al.  What's statistical about learning? Insights from modelling statistical learning as a set of memory processes , 2017, Philosophical Transactions of the Royal Society B: Biological Sciences.

[61]  Geoffrey E. Hinton Learning and Applying Contextual Constraints in Sentence Comprehension , 1991 .

[62]  Ullrich K. H. Ecker,et al.  Remembering perceptual features unequally bound in object and episodic tokens: Neural mechanisms and their electrophysiological correlates , 2010, Neuroscience & Biobehavioral Reviews.

[63]  R. O’Reilly,et al.  Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. , 2003, Psychological review.

[64]  Jelena Mirkovic,et al.  Incrementality and Prediction in Human Sentence Processing , 2009, Cogn. Sci..

[65]  Willard Van Orman Quine,et al.  Word and Object , 1960 .

[66]  Vladimir M. Sloutsky,et al.  From Perceptual Categories to Concepts: What Develops? , 2010, Cogn. Sci..

[67]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .