Neural representations of events arise from temporal community structure

Our experience of the world seems to divide naturally into discrete, temporally extended events, yet the mechanisms underlying the learning and identification of events are poorly understood. Research on event perception has focused on transient elevations in predictive uncertainty or surprise as the primary signal driving event segmentation. We present human behavioral and functional magnetic resonance imaging (fMRI) evidence in favor of a different account, in which event representations coalesce around clusters or 'communities' of mutually predicting stimuli. Through parsing behavior, fMRI adaptation and multivoxel pattern analysis, we demonstrate the emergence of event representations in a domain containing such community structure, but in which transition probabilities (the basis of uncertainty and surprise) are uniform. We present a computational account of how the relevant representations might arise, proposing a direct connection between event learning and the learning of semantic categories.

[1]  Darren Newtson Attribution and the unit of perception of ongoing behavior. , 1973 .

[2]  E. Rosch,et al.  Family resemblances: Studies in the internal structure of categories , 1975, Cognitive Psychology.

[3]  Douglas L. Medin,et al.  Context theory of classification learning. , 1978 .

[4]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[5]  James L. McClelland,et al.  Learning the structure of event sequences. , 1991, Journal of experimental psychology. General.

[6]  Y. Kareev,et al.  The emergence of events , 1994, Cognition.

[7]  Richard S. J. Frackowiak,et al.  Functional anatomy of a common semantic system for words and pictures , 1996, Nature.

[8]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[9]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[10]  A. Vinter,et al.  PARSER: A Model for Word Segmentation , 1998 .

[11]  E. Koechlin,et al.  Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[12]  Jeffrey M. Zacks,et al.  Human brain activity time-locked to perceptual event boundaries , 2001, Nature Neuroscience.

[13]  Jodie A. Baird,et al.  Making sense of human behavior: Action parsing and intentional inference , 2001 .

[14]  M E J Newman,et al.  Community structure in social and biological networks , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[15]  R. Aslin,et al.  Statistical learning of higher-order temporal structure from visual shape sequences. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[16]  Annie Vinter,et al.  The formation of structurally relevant units in artificial grammar learning , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[17]  Yasushi Miyashita,et al.  From Perception to Sentence Comprehension: The Convergence of Auditory and Visual Information of Language in the Left Inferior Frontal Cortex , 2002, NeuroImage.

[18]  S. Bookheimer,et al.  Dissociating Neural Mechanisms of Temporal Sequencing and Processing Phonemes , 2003, Neuron.

[19]  S. Thompson-Schill Neuroimaging studies of semantic memory: inferring “how” from “where” , 2003, Neuropsychologia.

[20]  Jeffery. M. Zacks,et al.  Activation of human motion processing areas during event perception , 2003, Cognitive, affective & behavioral neuroscience.

[21]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[22]  Karl Magnus Petersson,et al.  Artificial syntactic violations activate Broca's region , 2004, Cogn. Sci..

[23]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[24]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.

[25]  P. Fletcher,et al.  Selecting among competing alternatives: selection and retrieval in the left inferior frontal gyrus. , 2005, Cerebral cortex.

[26]  Jordan Grafman,et al.  Psychological structure and neural correlates of event knowledge. , 2005, Cerebral cortex.

[27]  Angela D. Friederici,et al.  Who did what to whom? The neural basis of argument hierarchies during language comprehension , 2005, NeuroImage.

[28]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[29]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Isabel Gauthier,et al.  Repetition‐induced changes in BOLD response reflect accumulation of neural activity , 2006, Human brain mapping.

[31]  M. Bar,et al.  The units of thought , 2007, Hippocampus.

[32]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[33]  Jonathan Berger,et al.  Neural Dynamics of Event Segmentation in Music: Converging Evidence for Dissociable Ventral and Dorsal Networks , 2007, Neuron.

[34]  Alana T. Wong,et al.  Remembering the past and imagining the future: Common and distinct neural substrates during event construction and elaboration , 2007, Neuropsychologia.

[35]  Jeffrey M. Zacks,et al.  A Computational Model of Event Segmentation From Perceptual Prediction , 2007, Cogn. Sci..

[36]  E. Pothos Theories of artificial grammar learning. , 2007, Psychological bulletin.

[37]  Andrew B. Leber,et al.  Visual quality determines the direction of neural repetition effects. , 2007, Cerebral cortex.

[38]  Dare A. Baldwin,et al.  Segmenting dynamic human action via statistical structure , 2008, Cognition.

[39]  Martin Rosvall,et al.  Maps of random walks on complex networks reveal community structure , 2007, Proceedings of the National Academy of Sciences.

[40]  Marvin M. Chun,et al.  Babies and Brains: Habituation in Infant Cognition and Functional Neuroimaging , 2008, Frontiers in human neuroscience.

[41]  Aron K Barbey,et al.  The frontopolar cortex mediates event knowledge complexity: a parametric functional MRI study , 2009, Neuroreport.

[42]  Karl J. Friston,et al.  Evidence of Mirror Neurons in Human Inferior Frontal Gyrus , 2009, The Journal of Neuroscience.

[43]  Santo Fortunato,et al.  Community detection in graphs , 2009, ArXiv.

[44]  Gilbert Remillard,et al.  Implicit learning of fifth- and sixth-order sequential probabilities , 2010, Memory & cognition.

[45]  Marc W. Howard,et al.  Constructing Semantic Representations From a Gradually Changing Representation of Temporal Context , 2011, Top. Cogn. Sci..

[46]  T. Rogers,et al.  Lichtheim 2: Synthesizing Aphasia and the Neural Basis of Language in a Neurocomputational Model of the Dual Dorsal-Ventral Language Pathways , 2011, Neuron.

[47]  Jeffrey M. Zacks,et al.  Prediction Error Associated with the Perceptual Segmentation of Naturalistic Events , 2011, Journal of Cognitive Neuroscience.

[48]  Francisco Pereira,et al.  Information mapping with pattern classifiers: A comparative study , 2011, NeuroImage.

[49]  L. Davachi,et al.  What Constitutes an Episode in Episodic Memory? , 2011, Psychological science.

[50]  Nikolaus Weiskopf,et al.  Dissociable roles of human inferior frontal gyrus during action execution and observation , 2012, NeuroImage.

[51]  Lauren V. Kustner,et al.  Shaping of Object Representations in the Human Medial Temporal Lobe Based on Temporal Regularities , 2012, Current Biology.