A Hierarchy of Functional States in Working Memory

Extensive research has examined how information is maintained in working memory (WM), but it remains unknown how WM is used to guide behaviour. We addressed this question by combining human electrophysiology with pattern analyses, cognitive modelling, and a task requiring maintenance of two WM items and priority shifts between them. This enabled us to discern neural states coding for immediately and prospectively task-relevant items, and to examine their contribution to WM-based decisions. We identified two qualitatively different states: a functionally active state encoded only immediately task-relevant items and closely tracked the quality of evidence integration on the current trial. In contrast, prospectively relevant items were encoded in a functionally latent state that did not engage with ongoing processing but tracked memory precision at longer time scales. These results delineate a hierarchy of functional states, whereby latent memories supporting general maintenance are transformed into active decision-circuits to guide flexible behaviour.

[1]  Walter J. Scheirer,et al.  PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  B. Postle,et al.  Different states of priority recruit different neural codes in visual working memory , 2018, bioRxiv.

[3]  Elkan G. Akyürek,et al.  Dynamic hidden states underlying working memory guided behaviour , 2017, Nature Neuroscience.

[4]  Gustavo Rohenkohl,et al.  Testing sensory evidence against mnemonic templates , 2015, eLife.

[5]  G. E. Alexander,et al.  Neuron Activity Related to Short-Term Memory , 1971, Science.

[6]  Christos Constantinidis,et al.  Stable population coding for working memory coexists with heterogeneous neural dynamics in prefrontal cortex , 2016, Proceedings of the National Academy of Sciences.

[7]  Benjamin J. Shannon,et al.  Parietal lobe contributions to episodic memory retrieval , 2005, Trends in Cognitive Sciences.

[8]  J. Gallant,et al.  Attention to Stimulus Features Shifts Spectral Tuning of V4 Neurons during Natural Vision , 2008, Neuron.

[9]  T. Ohshima,et al.  Stimulated emission from nitrogen-vacancy centres in diamond , 2016, Nature Communications.

[10]  W. Ma,et al.  Changing concepts of working memory , 2014, Nature Neuroscience.

[11]  A. Nobre,et al.  Prioritizing Information during Working Memory: Beyond Sustained Internal Attention , 2017, Trends in Cognitive Sciences.

[12]  Klaus Oberauer,et al.  Design for a working memory. , 2009 .

[13]  Olivier Ledoit,et al.  Honey, I Shrunk the Sample Covariance Matrix , 2003 .

[14]  Thomas V. Wiecki,et al.  HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python , 2013, Front. Neuroinform..

[15]  Klaus Oberauer,et al.  Neural Evidence for a Distinction between Short-term Memory and the Focus of Attention , 2012, Journal of Cognitive Neuroscience.

[16]  Timothy D. Hanks,et al.  Perceptual Decision Making in Rodents, Monkeys, and Humans , 2017, Neuron.

[17]  Edward F. Ester,et al.  Restoring Latent Visual Working Memory Representations in Human Cortex , 2016, Neuron.

[18]  P. Roelfsema,et al.  Different States in Visual Working Memory: When It Guides Attention and When It Does Not , 2022 .

[19]  P. Goldman-Rakic.  Cellular basis of working memory , 1995, Neuron.

[20]  Klaus Oberauer,et al.  Working Memory Load and the Retro-Cue Effect: A Diffusion Model Account , 2017, Journal of experimental psychology. Human perception and performance.

[21]  Daniel J Mitchell,et al.  Neural Coding for Instruction‐Based Task Sets in Human Frontoparietal and Visual Cortex , 2016, Cerebral cortex.

[22]  C. Olivers,et al.  The reliability of retro-cues determines the fate of noncued visual working memory representations , 2015, Psychonomic bulletin & review.

[23]  M. Chun.  Visual working memory as visual attention sustained internally over time , 2011, Neuropsychologia.

[24]  U. Rutishauser,et al.  Between persistently active and activity‐silent frameworks: novel vistas on the cellular basis of working memory , 2019, Annals of the New York Academy of Sciences.

[25]  Mark G. Stokes,et al.  Drifting codes within a stable coding scheme for working memory , 2019, bioRxiv.

[26]  Tirin Moore,et al.  Spatial working memory alters the efficacy of input to visual cortex , 2017, Nature Communications.

[27]  A. Nobre,et al.  Orienting Attention to Locations in Internal Representations , 2003, Journal of Cognitive Neuroscience.

[28]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[29]  M. Stokes.  ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework , 2015, Trends in Cognitive Sciences.

[30]  Masud Husain,et al.  Neural mechanisms of attending to items in working memory , 2019, Neuroscience & Biobehavioral Reviews.

[31]  Jan Kamiński,et al.  Intermediate-Term Memory as a Bridge between Working and Long-Term Memory , 2017, The Journal of Neuroscience.

[32]  John D E Gabrieli,et al.  Performance-Related Sustained and Anticipatory Activity in Human Medial Temporal Lobe during Delayed Match-to-Sample , 2009, The Journal of Neuroscience.

[33]  N. Sigala,et al.  Dynamic Coding for Cognitive Control in Prefrontal Cortex , 2013, Neuron.

[34]  Nicholas E. Myers,et al.  Benefits of Flexible Prioritization in Working Memory Can Arise Without Costs , 2017, Journal of experimental psychology. Human perception and performance.

[35]  G. D. Logan.  Task Switching , 2022 .

[36]  A. Hollingworth,et al.  Journal of Experimental Psychology : Human Perception and Performance Selective Maintenance in Visual Working Memory Does Not Require Sustained Visual Attention , 2012 .

[37]  Jarrod A. Lewis-Peacock,et al.  Multiple neural states of representation in short-term memory? It’s a matter of attention , 2014, Front. Hum. Neurosci..

[38]  Jack L. Gallant,et al.  Working Memory and Decision Processes in Visual Area V4 , 2013, Front. Neurosci..

[39]  Gi-Yeul Bae,et al.  What happens to an individual visual working memory representation when it is interrupted? , 2018, British journal of psychology.

[40]  T. Egner,et al.  Working memory as internal attention: Toward an integrative account of internal and external selection processes , 2012, Psychonomic Bulletin & Review.

[41]  Earl K. Miller,et al.  Working Memory 2.0 , 2018, Neuron.

[42]  Eric L. Denovellis,et al.  Synchronous Oscillatory Neural Ensembles for Rules in the Prefrontal Cortex , 2012, Neuron.

[43]  Tara Raam,et al.  Oxytocin-Sensitive Neurons in Prefrontal Cortex Gate Social Recognition Memory , 2020, The Journal of Neuroscience.

[44]  Klaus Oberauer,et al.  Getting more from visual working memory: Retro-cues enhance retrieval and protect from visual interference. , 2015, Journal of experimental psychology. Human perception and performance.

[45]  Joy J Geng,et al.  Distractor Probability Changes the Shape of the Attentional Template , 2017, Journal of experimental psychology. Human perception and performance.

[46]  Massimo Silvetti,et al.  Human midcingulate cortex encodes distributed representations of task progress , 2018, Proceedings of the National Academy of Sciences.

[47]  Zheng Liu,et al.  Short-Term Memory Trace in Rapidly Adapting Synapses of Inferior Temporal Cortex , 2008, PLoS Comput. Biol..

[48]  K. Oberauer.  Access to information in working memory: exploring the focus of attention. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[49]  C. Allefeld,et al.  Cortical specialization for attended versus unattended working memory , 2018, Nature Neuroscience.

[50]  M. D’Esposito.  Working memory. , 2008, Handbook of clinical neurology.

[51]  E. Miller,et al.  Gamma and Beta Bursts Underlie Working Memory , 2016, Neuron.

[52]  Adam C. Riggall,et al.  Reactivation of latent working memories with transcranial magnetic stimulation , 2016, Science.

[53]  Floris P de Lange,et al.  Prior expectations induce prestimulus sensory templates , 2017, Proceedings of the National Academy of Sciences.

[54]  M. Stokes,et al.  Theoretical distinction between functional states in working memory and their corresponding neural states , 2020, Visual cognition.

[55]  John Duncan,et al.  Goal neglect and Spearman's g: competing parts of a complex task. , 2008, Journal of experimental psychology. General.

[56]  Jarrod A. Lewis-Peacock,et al.  Neural Evidence for the Flexible Control of Mental Representations. , 2015, Cerebral cortex.

[57]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[58]  Etienne Koechlin,et al.  Foundations of human reasoning in the prefrontal cortex , 2014, Science.

[59]  Sammi R. Chekroud,et al.  Human gaze tracks attentional focusing in memorized visual space , 2019, Nature Human Behaviour.

[60]  Thomas C. Sprague,et al.  Functional MRI and EEG Index Complementary Attentional Modulations , 2019, The Journal of Neuroscience.

[61]  Johannes Jacobus Fahrenfort,et al.  Current and future goals are represented in opposite patterns in object-selective cortex , 2018, bioRxiv.

[62]  C. Curtis,et al.  Persistent activity in the prefrontal cortex during working memory , 2003, Trends in Cognitive Sciences.

[63]  A. Compte,et al.  Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory , 2014, Nature Neuroscience.

[64]  Klaus Oberauer,et al.  Decoding Attended Information in Short-term Memory: An EEG Study , 2013, Journal of Cognitive Neuroscience.

[65]  John Duncan,et al.  Goal neglect and knowledge chunking in the construction of novel behaviour☆ , 2014, Cognition.

[66]  Elkan G. Akyürek,et al.  Unimodal and Bimodal Access to Sensory Working Memories by Auditory and Visual Impulses , 2019, The Journal of Neuroscience.

[67]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[68]  Paul M Bays,et al.  Working memory retrieval as a decision process. , 2014, Journal of vision.

[69]  Christian Keysers,et al.  Using Bayes factor hypothesis testing in neuroscience to establish evidence of absence , 2020, Nature Neuroscience.

[70]  J. Serences,et al.  Adaptive Allocation of Attentional Gain , 2009, The Journal of Neuroscience.

[71]  Alessandra S. Souza,et al.  Retro-cue benefits in working memory without sustained focal attention , 2014, Memory & cognition.

[72]  Dimitrios Pantazis,et al.  Can visual information encoded in cortical columns be decoded from magnetoencephalography data in humans? , 2015, NeuroImage.