A Probabilistic Palimpsest Model of Visual Short-term Memory

Working memory plays a key role in cognition, and yet its mechanisms remain much debated. Human performance on memory tasks is severely limited; however, the two major classes of theory explaining the limits leave open questions about key issues such as how multiple simultaneously-represented items can be distinguished. We propose a palimpsest model, with the occurrent activity of a single population of neurons coding for several multi-featured items. Using a probabilistic approach to storage and recall, we show how this model can account for many qualitative aspects of existing experimental data. In our account, the underlying nature of a memory item depends entirely on the characteristics of the population representation, and we provide analytical and numerical insights into critical issues such as multiplicity and binding. We consider representations in which information about individual feature values is partially separate from the information about binding that creates single items out of multiple features. An appropriate balance between these two types of information is required to capture fully the different types of error seen in human experimental data. Our model provides the first principled account of misbinding errors. We also suggest a specific set of stimuli designed to elucidate the representations that subjects actually employ.

[1]  Paul M Bays,et al.  Dynamic Shifts of Limited Working Memory Resources in Human Vision , 2008, Science.

[2]  P. Cavanagh,et al.  The Capacity of Visual Short-Term Memory is Set Both by Visual Information Load and by Number of Objects , 2004, Psychological science.

[3]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[4]  Alexandre Pouget,et al.  Probabilistic Interpretation of Population Codes , 1996, Neural Computation.

[5]  Garrett Swan,et al.  The binding pool: A model of shared neural resources for distinct items in visual working memory , 2014, Attention, perception & psychophysics.

[6]  S. Luck,et al.  Sudden Death and Gradual Decay in Visual Working Memory , 2009, Psychological science.

[7]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[8]  Terrence J. Sejnowski,et al.  Neuronal Tuning: To Sharpen or Broaden? , 1999, Neural Computation.

[9]  H Sompolinsky,et al.  Simple models for reading neuronal population codes. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Timothy F. Brady,et al.  Encoding higher-order structure in visual working memory: A probabilistic model , 2010 .

[11]  Wei Ji Ma,et al.  No capacity limit in attentional tracking: evidence for probabilistic inference under a resource constraint. , 2009, Journal of vision.

[12]  Christopher L. Asplund,et al.  What are the units of storage in visual working memory? , 2010, Journal of vision.

[13]  Peter Dayan,et al.  The Effect of Correlated Variability on the Accuracy of a Population Code , 1999, Neural Computation.

[14]  Christian W. Eurich,et al.  Multidimensional Encoding Strategy of Spiking Neurons , 2000, Neural Computation.

[15]  George A Alvarez,et al.  Object-based benefits without object-based representations. , 2012, Journal of experimental psychology. General.

[16]  L. Abbott,et al.  Cascade Models of Synaptically Stored Memories , 2005, Neuron.

[17]  Alexander S. Ecker,et al.  Reassessing optimal neural population codes with neurometric functions , 2011, Proceedings of the National Academy of Sciences.

[18]  Aaron C. Courville,et al.  The pigeon as particle filter , 2007, NIPS 2007.

[19]  R. Zemel,et al.  Inference and computation with population codes. , 2003, Annual review of neuroscience.

[20]  Timothy F. Brady,et al.  Compression in visual working memory: using statistical regularities to form more efficient memory representations. , 2009, Journal of experimental psychology. General.

[21]  Boris S. Gutkin,et al.  Dopamine modulation in the basal ganglia locks the gate to working memory , 2006, Journal of Computational Neuroscience.

[22]  Mark Wexler,et al.  The nonlinear structure of motion perception during smooth eye movements. , 2009, Journal of vision.

[23]  D. Knill,et al.  Bayesian sampling in visual perception , 2011, Proceedings of the National Academy of Sciences.

[24]  Jeffrey S. Johnson,et al.  A layered neural architecture for the consolidation, maintenance, and updating of representations in visual working memory , 2009, Brain Research.

[25]  Melonie Williams,et al.  Directed forgetting and directed remembering in visual working memory. , 2012, Journal of experimental psychology. Learning, memory, and cognition.

[26]  George A. Alvarez,et al.  Object-Based Benefits without Object-Based Representations , 2012 .

[27]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[28]  Emma Y. Wu,et al.  Storage and binding of object features in visual working memory , 2011, Neuropsychologia.

[29]  Earl K Miller,et al.  The representation of multiple objects in prefrontal neuronal delay activity. , 2007, Cerebral cortex.

[30]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[31]  W. Ma,et al.  Factorial comparison of working memory models. , 2014, Psychological review.

[32]  W. Ma,et al.  A detection theory account of change detection. , 2004, Journal of vision.

[33]  József Fiser,et al.  Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment , 2011, Science.

[34]  Nancy B. Carlisle,et al.  The benefit of forgetting , 2012, Psychonomic Bulletin & Review.

[35]  Yaoda Xu,et al.  Encoding color and shape from different parts of an object in visual short-term memory , 2002, Perception & psychophysics.

[36]  George A. Alvarez,et al.  Compression in visual short-term memory: Using statistical regularities to form more efficient memory representations , 2010 .

[37]  Markus Siegel,et al.  Neural substrates of cognitive capacity limitations , 2011, Proceedings of the National Academy of Sciences.

[38]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[39]  G. Alvarez,et al.  Object features fail independently in visual working memory: evidence for a probabilistic feature-store model. , 2011, Journal of vision.

[40]  Xiao-Jing Wang Synaptic reverberation underlying mnemonic persistent activity , 2001, Trends in Neurosciences.

[41]  Peter Dayan,et al.  Doubly Distributional Population Codes: Simultaneous Representation of Uncertainty and Multiplicity , 2003, Neural Computation.

[42]  Wei Ji Ma,et al.  Variability in encoding precision accounts for visual short-term memory limitations , 2012, Proceedings of the National Academy of Sciences.

[43]  Harshinder Singh,et al.  Probabilistic model for two dependent circular variables , 2002 .

[44]  D. J. C. MacKay Maximum Entropy Connections: Neural Networks , 1991 .

[45]  Paul M Bays,et al.  Temporal dynamics of encoding, storage, and reallocation of visual working memory. , 2011, Journal of vision.

[46]  A. Pouget,et al.  Probabilistic population codes and the exponential family of distributions. , 2007, Progress in brain research.

[47]  Neil Burgess,et al.  Part or parcel? Contextual binding of events in episodic memory , 2006, Handbook of Binding and Memory.

[48]  Paul M Bays,et al.  Dynamic Updating of Working Memory Resources for Visual Objects , 2011, The Journal of Neuroscience.

[49]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[50]  Wei Ji Ma,et al.  No Evidence for an Item Limit in Change Detection , 2013, PLoS Comput. Biol..

[51]  R. Jacobs,et al.  An ideal observer analysis of visual working memory. , 2012, Psychological review.

[52]  P. Dayan,et al.  Matching storage and recall: hippocampal spike timing–dependent plasticity and phase response curves , 2005, Nature Neuroscience.

[53]  Sophie Denève,et al.  Spike-Based Population Coding and Working Memory , 2011, PLoS Comput. Biol..

[54]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[55]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[56]  George A. Alvarez,et al.  Variability in the quality of visual working memory , 2012, Nature Communications.

[57]  D. Perrett,et al.  The `Ideal Homunculus': decoding neural population signals , 1998, Trends in Neurosciences.

[58]  Robert A. Jacobs,et al.  A Nonparametric Bayesian Model of Visual Short-Term Memory , 2011, CogSci.

[59]  Jeffrey S. Johnson,et al.  Moving to higher ground: The dynamic field theory and the dynamics of visual cognition. , 2008, New ideas in psychology.

[60]  E. Koechlin,et al.  Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making , 2012, PLoS biology.

[61]  H Pashler,et al.  Familiarity and visual change detection , 1988, Perception & psychophysics.

[62]  Paul R. Martin,et al.  Receptive field asymmetries produce color-dependent direction selectivity in primate lateral geniculate nucleus. , 2010, Journal of vision.

[63]  R. Jacobs,et al.  A probabilistic clustering theory of the organization of visual short-term memory. , 2013, Psychological review.

[64]  Andreas Krause,et al.  Advances in Neural Information Processing Systems (NIPS) , 2014 .

[65]  S. Luck,et al.  Discrete fixed-resolution representations in visual working memory , 2008, Nature.

[66]  Xiao-Jing Wang,et al.  From Distributed Resources to Limited Slots in Multiple-Item Working Memory: A Spiking Network Model with Normalization , 2012, The Journal of Neuroscience.

[67]  S. Walker Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[68]  Robert A. Jacobs,et al.  Probabilistic Modeling of Dependencies Among Visual Short-Term Memory Representations , 2011, NIPS.

[69]  Edward Awh,et al.  Retraction Note to: The plateau in mnemonic resolution across large set sizes indicates discrete resource limits in visual working memory , 2012, Attention, perception & psychophysics.

[70]  Yuhong V. Jiang,et al.  Is visual short-term memory object based? Rejection of the “strong-object” hypothesis , 2002, Perception & psychophysics.

[71]  Stanislas Dehaene,et al.  Networks of Formal Neurons and Memory Palimpsests , 1986 .

[72]  Radford M. Neal Slice Sampling , 2003, The Annals of Statistics.

[73]  Kanti V. Mardia,et al.  A multivariate von mises distribution with applications to bioinformatics , 2008 .

[74]  Timothy F. Brady,et al.  A probabilistic model of visual working memory: Incorporating higher order regularities into working memory capacity estimates. , 2013, Psychological review.

[75]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[76]  Yaoda Xu,et al.  Limitations of object-based feature encoding in visual short-term memory. , 2002, Journal of experimental psychology. Human perception and performance.

[77]  Haim Sompolinsky,et al.  The Effect of Correlations on the Fisher Information of Population Codes , 1998, NIPS.

[78]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[79]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[80]  P. Földiák,et al.  The ‘Ideal Homunculus’: Statistical Inference from Neural Population Responses , 1993 .

[81]  M. Bethge,et al.  Mixtures of Conditional Gaussian Scale Mixtures Applied to Multiscale Image Representations , 2011, PloS one.

[82]  Yaoda Xu,et al.  Feature Integration across Parts in Visual Search , 2002, Perception.

[83]  Paul M Bays,et al.  The precision of visual working memory is set by allocation of a shared resource. , 2009, Journal of vision.

[84]  Peter Dayan,et al.  Two is better than one: distinct roles for familiarity and recollection in retrieving palimpsest memories , 2011, NIPS.

[85]  Paul M Bays,et al.  Noise in Neural Populations Accounts for Errors in Working Memory , 2014, The Journal of Neuroscience.

[86]  Aapo Hyvärinen,et al.  Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior , 2002, NIPS.

[87]  Nicolas Brunel,et al.  Mutual Information, Fisher Information, and Population Coding , 1998, Neural Computation.