Sequence structure organizes items in varied latent states of working memory neural network

In memory experiences, events do not exist independently but are linked with each other via structure-based organization. Structure knowledge largely influences memory behavior, but how it is implemented in the brain remains unknown. Here, we combined magnetoencephalogram (MEG) recordings, computational modeling, and impulse-response approaches to probe the latent states when subjects held a list of items in working memory (WM). We demonstrate that sequence structure reorganizes WM items into distinct latent states, i.e., being reactivated at different latencies, and the reactivation profiles further correlate with recency behavior. In contrast, memorizing the same list of items without sequence requirements disrupts the recency effect and elicits comparable reactivations. Finally, computational modeling reveals a dominant function of high-level representations that characterize the abstract sequence structure, instead of low-level information decaying, in mediating sequence memory. Taken together, sequence structure shapes the way WM items are stored in the brain and essentially influences memory behavior.

[1]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[2]  M. Tsodyks,et al.  A theory of working memory without consciousness or sustained activity , 2016, bioRxiv.

[3]  Adam C. Riggall,et al.  Reactivation of latent working memories with transcranial magnetic stimulation , 2016, Science.

[4]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[5]  Lenny A. Varghese,et al.  On the utility of perceptual anchors during pure-tone frequency discrimination. , 2020, The Journal of the Acoustical Society of America.

[6]  A. Baddeley Working memory: looking back and looking forward , 2003, Nature Reviews Neuroscience.

[7]  D. Navon Forest before trees: The precedence of global features in visual perception , 1977, Cognitive Psychology.

[8]  Earl K. Miller,et al.  Working Memory 2.0 , 2018, Neuron.

[9]  S. Hochstein,et al.  The reverse hierarchy theory of visual perceptual learning , 2004, Trends in Cognitive Sciences.

[10]  Jonathan I. Flombaum,et al.  Why some colors appear more memorable than others: A model combining categories and particulars in color working memory. , 2015, Journal of experimental psychology. General.

[11]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[12]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[13]  L. Davachi,et al.  The influence of context boundaries on memory for the sequential order of events. , 2013, Journal of experimental psychology. General.

[14]  A. Hupbach,et al.  Neural Context Reinstatement Predicts Memory Misattribution , 2013, The Journal of Neuroscience.

[15]  S. Taulu,et al.  Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements , 2006, Physics in medicine and biology.

[16]  B. Postle,et al.  Different states of priority recruit different neural codes in visual working memory , 2018, bioRxiv.

[17]  Zeb Kurth-Nelson,et al.  Fast Sequences of Non-spatial State Representations in Humans , 2016, Neuron.

[18]  Timothy F. Brady,et al.  Hierarchical Encoding in Visual Working Memory , 2010, Psychological science.

[19]  M. Tsodyks,et al.  Synaptic Theory of Working Memory , 2008, Science.

[20]  David S. Lorberbaum,et al.  Genetic evidence that Nkx2.2 acts primarily downstream of Neurog3 in pancreatic endocrine lineage development , 2017, eLife.

[21]  Floris P de Lange,et al.  Prior expectations induce prestimulus sensory templates , 2017, Proceedings of the National Academy of Sciences.

[22]  Marten Postma,et al.  PlotsOfData—A web app for visualizing data together with their summaries , 2019, PLoS biology.

[23]  D. Heeger,et al.  Decoding and Reconstructing Color from Responses in Human Visual Cortex , 2009, The Journal of Neuroscience.

[24]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[25]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[26]  Zeb Kurth-Nelson,et al.  What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior , 2018, Neuron.

[27]  Jianrong Jia,et al.  Fast-backward replay of sequentially memorized items in humans , 2018, bioRxiv.

[28]  Yoshua Bengio,et al.  Better Mixing via Deep Representations , 2012, ICML.

[29]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[30]  Edward Awh,et al.  Item-specific delay activity demonstrates concurrent storage of multiple active neural representations in working memory , 2019, PLoS biology.

[31]  D. Heeger,et al.  Cross-orientation suppression in human visual cortex. , 2011, Journal of neurophysiology.

[32]  Bart Gips,et al.  Temporal coding organized by coupled alpha and gamma oscillations prioritize visual processing , 2014, Trends in Neurosciences.

[33]  Misha Tsodyks,et al.  Visual perception as retrospective Bayesian decoding from high- to low-level features , 2017, Proceedings of the National Academy of Sciences.

[34]  J. Harris,et al.  The decline of pitch discrimination with time. , 1952, Journal of experimental psychology.

[35]  Mark G. Stokes,et al.  Drifting codes within a stable coding scheme for working memory , 2019, bioRxiv.

[36]  J. L. de la Pompa,et al.  A novel source of arterial valve cells linked to bicuspid aortic valve without raphe in mice , 2018, eLife.

[37]  Donald E. Broadbent,et al.  Recency Effects in Visual Memory , 1981 .

[38]  Edward F. Ester,et al.  Parietal and Frontal Cortex Encode Stimulus-Specific Mnemonic Representations during Visual Working Memory , 2015, Neuron.

[39]  John T. Serences,et al.  Reconstructions of Information in Visual Spatial Working Memory Degrade with Memory Load , 2014, Current Biology.

[40]  C. Curtis,et al.  Persistent activity in the prefrontal cortex during working memory , 2003, Trends in Cognitive Sciences.

[41]  R. VanRullen,et al.  An oscillatory mechanism for prioritizing salient unattended stimuli , 2012, Trends in Cognitive Sciences.

[42]  L Chen,et al.  Topological structure in visual perception. , 1982, Science.

[43]  M. Stokes ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework , 2015, Trends in Cognitive Sciences.

[44]  Timothy E. J. Behrens,et al.  Human Replay Spontaneously Reorganizes Experience , 2019, Cell.

[45]  R. Jackendoff What is a cognitive map? , 1979, Behavioral and Brain Sciences.

[46]  Edward F. Ester,et al.  Restoring Latent Visual Working Memory Representations in Human Cortex , 2016, Neuron.

[47]  Elkan G. Akyürek,et al.  Dynamic hidden states underlying working memory guided behaviour , 2017, Nature Neuroscience.

[48]  Gustavo Rohenkohl,et al.  Testing sensory evidence against mnemonic templates , 2015, eLife.

[49]  Wei Ji Ma,et al.  Bayesian Decision Models: A Primer , 2019, Neuron.

[50]  P. Goldman-Rakic Cellular basis of working memory , 1995, Neuron.