Mental compression of spatial sequences in human working memory using numerical and geometrical primitives

Abstract How do humans encode spatial and sequential information in working memory? We tested the hypothesis that participants do not merely store each consecutive location in a distinct memory slot, but instead compress the whole sequence using an abstract language-like code that captures geometrical regularities at multiple nested levels. We exposed participants to sequences of fixed length but variable regularity, while their brain activity was recorded using magneto-encephalography. The entire sequence could be decoded from brain signals. Anticipation signals could also be decoded, and both behavior and anticipatory brain signals were modulated by sequence complexity, defined as the minimal description length provided by the formal language. Furthermore, abstract primitives of rotation and symmetry could be decoded both in isolation and within the sequences. These results suggest that humans encode the transitions between sequence items as rotations and symmetries, and compress longer sequences using nested repetitions of those primitives.

[1]  Stanislas Dehaene,et al.  Cortical circuits for mathematical knowledge: evidence for a major subdivision within the brain's semantic networks , 2018, Philosophical Transactions of the Royal Society B: Biological Sciences.

[2]  G. Orban,et al.  Processing of Abstract Ordinal Knowledge in the Horizontal Segment of the Intraparietal Sulcus , 2007, The Journal of Neuroscience.

[3]  C. Lebiere,et al.  An integrated theory of list memory. , 1998 .

[4]  Christopher I. Petkov,et al.  Auditory and Visual Sequence Learning in Humans and Monkeys using an Artificial Grammar Learning Paradigm , 2017, Neuroscience.

[5]  S. Taulu,et al.  Suppression of Interference and Artifacts by the Signal Space Separation Method , 2003, Brain Topography.

[6]  B. Murdock,et al.  Memory for Serial Order , 1989 .

[7]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[8]  H. Eichenbaum,et al.  Evolution of declarative memory , 2006, Hippocampus.

[9]  Anke Marit Albers,et al.  Eye Movement-Related Confounds in Neural Decoding of Visual Working Memory Representations , 2017, eNeuro.

[10]  S. Maier,et al.  CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE 227 The Simplicity Principle in Human Concept Learning , 2022 .

[11]  E. Koechlin,et al.  Broca's Area and the Hierarchical Organization of Human Behavior , 2006, Neuron.

[12]  Mariano Sigman,et al.  LT^2C^2: A language of thought with Turing-computable Kolmogorov complexity , 2013, ArXiv.

[13]  J. Tenenbaum,et al.  Word learning as Bayesian inference. , 2007, Psychological review.

[14]  Noam Chomsky,et al.  The faculty of language: what is it, who has it, and how did it evolve? , 2002 .

[15]  M. D’Esposito,et al.  Is the rostro-caudal axis of the frontal lobe hierarchical? , 2009, Nature Reviews Neuroscience.

[16]  Martin Luessi,et al.  MEG and EEG data analysis with MNE-Python , 2013, Front. Neuroinform..

[17]  J. Duncan,et al.  Encoding Strategies Dissociate Prefrontal Activity from Working Memory Demand , 2003, Neuron.

[18]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[19]  Marcel A. J. van Gerven,et al.  Eye movements explain decodability during perception and cued attention in MEG , 2019, NeuroImage.

[20]  S. Dehaene,et al.  Representation of number in the brain. , 2009, Annual review of neuroscience.

[21]  W. Fitch,et al.  Non-adjacent visual dependency learning in chimpanzees , 2015, Animal Cognition.

[22]  David Badre,et al.  Frontal Cortex and the Hierarchical Control of Behavior , 2018, Trends in Cognitive Sciences.

[23]  Liping Wang,et al.  Large-Scale Cortical Networks for Hierarchical Prediction and Prediction Error in the Primate Brain , 2018, Neuron.

[24]  Alan D Baddeley,et al.  Memory for serial order across domains: An overview of the literature and directions for future research. , 2014, Psychological bulletin.

[25]  Michael Leyton,et al.  A generative theory of shape , 2004, Proceedings Shape Modeling Applications, 2004..

[26]  Paul Smolensky,et al.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1990, Artif. Intell..

[27]  Jacob Feldman,et al.  Minimization of Boolean complexity in human concept learning , 2000, Nature.

[28]  Dmitriy Aronov,et al.  Mapping of a non-spatial dimension by the hippocampal/entorhinal circuit , 2017, Nature.

[29]  Andreas Nieder,et al.  Supramodal numerosity selectivity of neurons in primate prefrontal and posterior parietal cortices , 2012, Proceedings of the National Academy of Sciences.

[30]  Timothy E. J. Behrens,et al.  Organizing conceptual knowledge in humans with a gridlike code , 2016, Science.

[31]  Dirk Abel,et al.  Simulation physiologischer Regelkreise mit der objektorientierten Modellbibliothek “HumanLib” , 2011, Autom..

[32]  Pierre Pica,et al.  Core knowledge of geometry in an Amazonian indigene group. , 2006, Science.

[33]  Yukiko Kikuchi,et al.  Auditory Artificial Grammar Learning in Macaque and Marmoset Monkeys , 2013, The Journal of Neuroscience.

[34]  N. Chater,et al.  Simplicity: a unifying principle in cognitive science? , 2003, Trends in Cognitive Sciences.

[35]  Shaul Hochstein,et al.  Macaque monkeys categorize images by their ordinal number , 2000, Nature.

[36]  Matthew Botvinick,et al.  From Numerosity to Ordinal Rank: A Gain-Field Model of Serial Order Representation in Cortical Working Memory , 2007, The Journal of Neuroscience.

[37]  S. Dehaene,et al.  Brain Mechanisms Underlying the Brief Maintenance of Seen and Unseen Sensory Information , 2016, Neuron.

[38]  M. Hauser,et al.  Segmentation of the speech stream in a non-human primate: statistical learning in cotton-top tamarins , 2001, Cognition.

[39]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[40]  Y. Miyashita,et al.  Neural organization for the long-term memory of paired associates , 1991, Nature.

[41]  M. Buiatti,et al.  Electrophysiological evidence of statistical learning of long-distance dependencies in 8-month-old preterm and full-term infants , 2015, Brain and Language.

[42]  Stanislas Dehaene,et al.  Neurophysiological dynamics of phrase-structure building during sentence processing , 2017, Proceedings of the National Academy of Sciences.

[43]  Claudia Männel,et al.  Evolutionary origins of non-adjacent sequence processing in primate brain potentials , 2016, Scientific Reports.

[44]  W Tecumseh Fitch,et al.  Toward a computational framework for cognitive biology: unifying approaches from cognitive neuroscience and comparative cognition. , 2014, Physics of life reviews.

[45]  Elizabeth M. Brannon,et al.  Serial Expertise of Rhesus Macaques , 2003, Psychological science.

[46]  D. Poeppel,et al.  Cortical Tracking of Hierarchical Linguistic Structures in Connected Speech , 2015, Nature Neuroscience.

[47]  G. Boole An Investigation of the Laws of Thought: On which are founded the mathematical theories of logic and probabilities , 2007 .

[48]  David J. Freedman,et al.  Emergence of abstract rules in the primate brain , 2020, Nature reviews. Neuroscience.

[49]  William D. Marslen-Wilson,et al.  Conserved Sequence Processing in Primate Frontal Cortex , 2017, Trends in Neurosciences.

[50]  J. Tanji,et al.  Integration of temporal order and object information in the monkey lateral prefrontal cortex. , 2004, Journal of neurophysiology.

[51]  Floris P de Lange,et al.  Prior expectations induce prestimulus sensory templates , 2017, Proceedings of the National Academy of Sciences.

[52]  Peter M. Vishton,et al.  Rule learning by seven-month-old infants. , 1999, Science.

[53]  Mariano Sigman,et al.  The cortical representation of simple mathematical expressions , 2012, NeuroImage.

[54]  Wen Fang,et al.  Representation of spatial sequences using nested rules in human prefrontal cortex , 2019, NeuroImage.

[55]  David Badre,et al.  Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes , 2008, Trends in Cognitive Sciences.

[56]  Y. Miyashita,et al.  Top-down signal from prefrontal cortex in executive control of memory retrieval , 1999, Nature.

[57]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[58]  W. Tecumseh Fitch,et al.  Action at a distance: dependency sensitivity in a New World primate , 2013, Biology Letters.

[59]  Nathan Weisz,et al.  Automatic and feature-specific prediction-related neural activity in the human auditory system , 2019, Nature Communications.

[60]  Sang Ah Lee,et al.  Beyond Core Knowledge: Natural Geometry , 2010, Cogn. Sci..

[61]  Caswell Barry,et al.  The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation , 2019, Cell.

[62]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[63]  Karl J. Friston,et al.  Dynamic causal modelling , 2003, NeuroImage.

[64]  Robert A Jacobs,et al.  Learning multisensory representations for auditory-visual transfer of sequence category knowledge: a probabilistic language of thought approach , 2014, Psychonomic bulletin & review.

[65]  Mariano Sigman,et al.  Bayesian validation of grammar productions for the language of thought , 2018, PloS one.

[66]  C. Summerfield,et al.  Expectation in perceptual decision making: neural and computational mechanisms , 2014, Nature Reviews Neuroscience.

[67]  A. Baddeley Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[68]  Jessica F. Cantlon,et al.  Recursive sequence generation in monkeys, children, U.S. adults, and native Amazonians , 2020, Science Advances.

[69]  P. Roelfsema,et al.  Incremental grouping of image elements in vision , 2011, Attention, perception & psychophysics.

[70]  Timothy E. J. Behrens,et al.  Human Replay Spontaneously Reorganizes Experience , 2019, Cell.

[71]  Alexandre Gramfort,et al.  A Reproducible MEG/EEG Group Study With the MNE Software: Recommendations, Quality Assessments, and Good Practices , 2018, Front. Neurosci..

[72]  Theresa M. Desrochers,et al.  The Necessity of Rostrolateral Prefrontal Cortex for Higher-Level Sequential Behavior , 2015, Neuron.

[73]  Florent Meyniel,et al.  The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees , 2015, Neuron.

[74]  M. Hauser,et al.  Learning at a distance II. Statistical learning of non-adjacent dependencies in a non-human primate , 2004, Cognitive Psychology.

[75]  D. McDermott LANGUAGE OF THOUGHT , 2012 .

[76]  S. Dehaene,et al.  Representation of Numerical and Sequential Patterns in Macaque and Human Brains , 2015, Current Biology.

[77]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[78]  Janneke F. M. Jehee,et al.  Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex , 2012, Neuron.

[79]  Stanislas Dehaene,et al.  Production of Supra-regular Spatial Sequences by Macaque Monkeys , 2018, Current Biology.

[80]  W. Fitch,et al.  Computational Constraints on Syntactic Processing in a Nonhuman Primate , 2004, Science.

[81]  Florian Mormann,et al.  Single Neurons in the Human Brain Encode Numbers , 2018, Neuron.

[82]  C. Gilbert,et al.  Contour Saliency in Primary Visual Cortex , 2006, Neuron.

[83]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[84]  Floris P de Lange,et al.  Time-compressed preplay of anticipated events in human primary visual cortex , 2017, Nature Communications.

[85]  E. Koechlin,et al.  The Architecture of Cognitive Control in the Human Prefrontal Cortex , 2003, Science.

[86]  R. Shepard,et al.  Learning and memorization of classifications. , 1961 .

[87]  J. Saffran Statistical Language Learning , 2003 .

[88]  Matthew M Botvinick,et al.  Short-term memory for serial order: a recurrent neural network model. , 2006, Psychological review.

[89]  Thomas D Albright,et al.  On the Perception of Probable Things: Neural Substrates of Associative Memory, Imagery, and Perception , 2012, Neuron.

[90]  Andreas Nieder,et al.  Temporal and Spatial Enumeration Processes in the Primate Parietal Cortex , 2006, Science.

[91]  M. D’Esposito,et al.  Frontal Cortex and the Discovery of Abstract Action Rules , 2010, Neuron.

[92]  C. Gilbert,et al.  Top-down influences on visual processing , 2013, Nature Reviews Neuroscience.

[93]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[94]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[95]  F. Mathy,et al.  What’s magic about magic numbers? Chunking and data compression in short-term memory , 2012, Cognition.

[96]  Mariano Sigman,et al.  The language of geometry: Fast comprehension of geometrical primitives and rules in human adults and preschoolers , 2017, PLoS Comput. Biol..

[97]  Michael Siegal,et al.  Agrammatic but numerate. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[98]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[99]  Floris P. de Lange,et al.  Prior Expectations Evoke Stimulus Templates in the Primary Visual Cortex , 2014, Journal of Cognitive Neuroscience.