A Flexible Model of Working Memory

Working memory is fundamental to cognition, allowing one to hold information "in mind." A defining characteristic of working memory is its flexibility: we can hold anything in mind. However, typical models of working memory rely on finely tuned, content-specific attractors to persistently maintain neural activity and therefore do not allow for the flexibility observed in behavior. Here, we present a flexible model of working memory that maintains representations through random recurrent connections between two layers of neurons: a structured "sensory" layer and a randomly connected, unstructured layer. As the interactions are untuned with respect to the content being stored, the network maintains any arbitrary input. However, in our model, this flexibility comes at a cost: the random connections overlap, leading to interference between representations and limiting the memory capacity of the network. Additionally, our model captures several other key behavioral and neurophysiological characteristics of working memory.

[1]  K. Harris,et al.  Cortical connectivity and sensory coding , 2013, Nature.

[2]  Paul M Bays,et al.  Dynamic Shifts of Limited Working Memory Resources in Human Vision , 2008, Science.

[3]  Garrett Swan,et al.  The binding pool: A model of shared neural resources for distinct items in visual working memory , 2014, Attention, perception & psychophysics.

[4]  Henning Sprekeler,et al.  Inhibitory Plasticity Balances Excitation and Inhibition in Sensory Pathways and Memory Networks , 2011, Science.

[5]  W. Ma,et al.  Changing concepts of working memory , 2014, Nature Neuroscience.

[6]  J. Duncan.  An adaptive coding model of neural function in prefrontal cortex , 2001 .

[7]  Nathan R. Wilson,et al.  Division and subtraction by distinct cortical inhibitory networks in vivo , 2012, Nature.

[8]  John T. Serences,et al.  Reconstructions of Information in Visual Spatial Working Memory Degrade with Memory Load , 2014, Current Biology.

[9]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[10]  Lynne Kiorpes,et al.  Visual development in primates: Neural mechanisms and critical periods , 2015, Developmental neurobiology.

[11]  P. J. Sjöström,et al.  Functional specificity of local synaptic connections in neocortical networks , 2011, Nature.

[12]  M. Hasselmo,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.11 Mechanisms underlying working memory for novel information , 2022 .

[13]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[14]  L. F. Abbott,et al.  Generating Coherent Patterns of Activity from Chaotic Neural Networks , 2009, Neuron.

[15]  Tim P Vogels,et al.  Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons , 2005, The Journal of Neuroscience.

[16]  Dmitri B. Chklovskii,et al.  Neuronal Circuits Underlying Persistent Representations Despite Time Varying Activity , 2012, Current Biology.

[17]  M. Sur,et al.  Invariant computations in local cortical networks with balanced excitation and inhibition , 2005, Nature Neuroscience.

[18]  Rex E. Jung,et al.  Structural brain variation and general intelligence , 2004, NeuroImage.

[19]  Rodrigo F. Salazar,et al.  Content-Specific Fronto-Parietal Synchronization During Visual Working Memory , 2012, Science.

[20]  P. Goldman-Rakic,et al.  Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. , 1989, Journal of neurophysiology.

[21]  R. Romo,et al.  Neuronal correlates of decision-making in secondary somatosensory cortex , 2002, Nature Neuroscience.

[22]  Eero P. Simoncelli,et al.  Computational models of cortical visual processing. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Anastasia Kiyonaga,et al.  Center-Surround Inhibition in Working Memory , 2016, Current Biology.

[24]  Julio C. Martinez-Trujillo,et al.  Sharp emergence of feature-selective sustained activity along the dorsal visual pathway , 2014, Nature Neuroscience.

[25]  M. Carandini,et al.  Parvalbumin-Expressing Interneurons Linearly Transform Cortical Responses to Visual Stimuli , 2012, Neuron.

[26]  Daniel P. Bliss,et al.  Serial Dependence across Perception, Attention, and Memory , 2017, Trends in Cognitive Sciences.

[27]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[28]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[29]  Maro G. Machizawa,et al.  Neural activity predicts individual differences in visual working memory capacity , 2004, Nature.

[30]  T. Yagi,et al.  Molecular diversity of clustered protocadherin-α required for sensory integration and short-term memory in mice , 2018, Scientific Reports.

[31]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[32]  Johannes D. Seelig,et al.  Neural dynamics for landmark orientation and angular path integration , 2015, Nature.

[33]  V. Jayaraman,et al.  Ring attractor dynamics in the Drosophila central brain , 2017, Science.

[34]  Paul M Bays,et al.  The precision of visual working memory is set by allocation of a shared resource. , 2009, Journal of vision.

[35]  Xiao-Jing Wang.  Synaptic reverberation underlying mnemonic persistent activity , 2001, Trends in Neurosciences.

[36]  M. Stokes.  ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework , 2015, Trends in Cognitive Sciences.

[37]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[38]  Dimitar Kostadinov,et al.  Protocadherin-dependent dendritic self-avoidance regulates neural connectivity and circuit function , 2015, eLife.

[39]  Jesper Tegnér,et al.  Mechanism for top-down control of working memory capacity , 2009, Proceedings of the National Academy of Sciences.

[40]  T. Klingberg.  Development of a superior frontal–intraparietal network for visuo-spatial working memory , 2006, Neuropsychologia.

[41]  P. Roelfsema,et al.  The Distributed Nature of Working Memory , 2017, Trends in Cognitive Sciences.

[42]  L. Abbott,et al.  Neural network dynamics. , 2005, Annual review of neuroscience.

[43]  Edward Awh,et al.  Clear evidence for item limits in visual working memory , 2017, Cognitive Psychology.

[44]  Stefano Fusi,et al.  Why neurons mix: high dimensionality for higher cognition , 2016, Current Opinion in Neurobiology.

[45]  R. Yuste,et al.  Dense, Unspecific Connectivity of Neocortical Parvalbumin-Positive Interneurons: A Canonical Microcircuit for Inhibition? , 2011, The Journal of Neuroscience.

[46]  Monica Melby-Lervåg,et al.  Is working memory training effective? A meta-analytic review. , 2013, Developmental psychology.

[47]  Masud Husain,et al.  Rapid Forgetting Results From Competition Over Time Between Items in Visual Working Memory , 2016, Journal of experimental psychology. Learning, memory, and cognition.

[48]  M. Tsodyks,et al.  Working models of working memory , 2014, Current Opinion in Neurobiology.

[49]  Anirvan Ghosh,et al.  Specification of synaptic connectivity by cell surface interactions , 2015, Nature Reviews Neuroscience.

[50]  Su Z. Hong,et al.  Distinct Eligibility Traces for LTP and LTD in Cortical Synapses , 2015, Neuron.

[51]  C. Curtis,et al.  Persistent activity in the prefrontal cortex during working memory , 2003, Trends in Cognitive Sciences.

[52]  H. Sompolinsky,et al.  Temporal integration by calcium dynamics in a model neuron , 2003, Nature Neuroscience.

[53]  Paul M Bays,et al.  Noise in Neural Populations Accounts for Errors in Working Memory , 2014, The Journal of Neuroscience.

[54]  N. Sigala,et al.  Dynamic Coding for Cognitive Control in Prefrontal Cortex , 2013, Neuron.

[55]  Markus Siegel,et al.  Neural substrates of cognitive capacity limitations , 2011, Proceedings of the National Academy of Sciences.

[56]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[57]  Christos Constantinidis,et al.  Stable population coding for working memory coexists with heterogeneous neural dynamics in prefrontal cortex , 2016, Proceedings of the National Academy of Sciences.

[58]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[59]  D. Heeger.  Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[60]  Hanna Damasio,et al.  Neural convergence and divergence in the mammalian cerebral cortex: From experimental neuroanatomy to functional neuroimaging , 2013, The Journal of comparative neurology.

[61]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[62]  T. Harkany,et al.  Pyramidal cell communication within local networks in layer 2/3 of rat neocortex , 2003, The Journal of physiology.

[63]  Frank Tong,et al.  Evidence of Gradual Loss of Precision for Simple Features and Complex Objects in Visual Working Memory , 2018, Journal of experimental psychology. Human perception and performance.

[64]  Julie C. Helmers,et al.  Chunking as a rational strategy for lossy data compression in visual working memory , 2017, bioRxiv.

[65]  P. Bays.  Spikes not slots: noise in neural populations limits working memory , 2015, Trends in Cognitive Sciences.

[66]  S. Luck,et al.  The influence of similarity on visual working memory representations , 2009, Visual cognition.

[67]  A. Compte,et al.  Neural circuit basis of visuo-spatial working memory precision: a computational and behavioral study. , 2015, Journal of neurophysiology.

[68]  M. Tsodyks,et al.  Synaptic Theory of Working Memory , 2008, Science.

[69]  E. Miller,et al.  Gamma and Beta Bursts Underlie Working Memory , 2016, Neuron.

[70]  Yoram Burak,et al.  Fundamental limits on persistent activity in networks of noisy neurons , 2012, Proceedings of the National Academy of Sciences.

[71]  Stefano Fusi,et al.  Hebbian Learning in a Random Network Captures Selectivity Properties of the Prefrontal Cortex , 2017, The Journal of Neuroscience.

[72]  R. Desimone,et al.  Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4 , 1999, The Journal of Neuroscience.

[73]  Adam J. Woods,et al.  Frontal Structural Neural Correlates of Working Memory Performance in Older Adults , 2017, Front. Aging Neurosci..

[74]  J. Fuster.  Memory in the cerebral cortex : an empirical approach to neural networks in the human and nonhuman primate , 1996 .

[75]  S. W. Kuffler.  Discharge patterns and functional organization of mammalian retina. , 1953, Journal of neurophysiology.

[76]  Romain Brette,et al.  Brian 2 - the second coming: spiking neural network simulation in Python with code generation , 2013, BMC Neuroscience.

[77]  M Petrides,et al.  Impairments on nonspatial self-ordered and externally ordered working memory tasks after lesions of the mid-dorsal part of the lateral frontal cortex in the monkey , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[78]  B. Postle.  Working Memory Functions of the Prefrontal Cortex , 2017 .

[79]  Blake S. Porter,et al.  Hippocampal Representation of Related and Opposing Memories Develop within Distinct, Hierarchically Organized Neural Schemas , 2014, Neuron.

[80]  C. Constantinidis,et al.  The neuroscience of working memory capacity and training , 2016, Nature Reviews Neuroscience.

[81]  T. Pasternak,et al.  Directional Signals in the Prefrontal Cortex and in Area MT during a Working Memory for Visual Motion Task , 2006, The Journal of Neuroscience.

[82]  Pierre Yger,et al.  Brian 2: neural simulations on a variety of computational hardware , 2014, BMC Neuroscience.

[83]  J. Fuster.  Unit activity in prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. , 1973, Journal of neurophysiology.

[84]  N. Cowan,et al.  The Magical Mystery Four , 2010, Current directions in psychological science.

[85]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[86]  S. Funahashi,et al.  Stable and Dynamic Coding for Working Memory in Primate Prefrontal Cortex , 2017, The Journal of Neuroscience.