Stochastic sampling provides a unifying account of working memory limits

Research into human working memory limits has been shaped by the competition between different formal models, with a central point of contention being whether internal representations are continuous or discrete. Here we describe a sampling approach derived from principles of neural coding as a new framework to understand working memory limits. Reconceptualizing existing models in these terms reveals strong commonalities between seemingly opposing accounts, but also allows us to identify specific points of difference. We show that the discrete versus continuous nature of sampling is not critical to model fits, but that instead random variability in sample counts is the key to reproducing human performance in both single- and whole-report tasks. A probabilistic limit on the number of items successfully retrieved is an emergent property of stochastic sampling, requiring no explicit mechanism to enforce it. These findings resolve discrepancies between previous accounts and establish a unified computational framework for working memory that is compatible with neural principles.

[1]  Alexander S. Ecker,et al.  Decorrelated Neuronal Firing in Cortical Microcircuits , 2010, Science.

[2]  Paul M Bays,et al.  Dynamic Shifts of Limited Working Memory Resources in Human Vision , 2008, Science.

[3]  Timothy F. Brady,et al.  Psychophysical scaling reveals a unified theory of visual memory strength , 2018, Nature Human Behaviour.

[4]  Wei Ji Ma,et al.  Variability in encoding precision accounts for visual short-term memory limitations , 2012, Proceedings of the National Academy of Sciences.

[5]  Peter Dayan,et al.  A Probabilistic Palimpsest Model of Visual Short-term Memory , 2015, PLoS Comput. Biol..

[6]  Christopher C. Pack,et al.  Bidirectional manipulation of GABAergic inhibition in MT: A comparison of neuronal and psychophysical performance , 2014 .

[7]  Stephen M. Emrich,et al.  Attention Mediates the Flexible Allocation of Visual Working Memory Resources , 2017, Journal of experimental psychology. Human perception and performance.

[8]  S. Morad,et al.  Ceramide-orchestrated signalling in cancer cells , 2012, Nature Reviews Cancer.

[9]  W. Ma,et al.  Factorial comparison of working memory models. , 2014, Psychological review.

[10]  W. Ma,et al.  A detection theory account of change detection. , 2004, Journal of vision.

[11]  Sebastian Schneegans,et al.  Neural Architecture for Feature Binding in Visual Working Memory , 2017, The Journal of Neuroscience.

[12]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[13]  Simon Farrell,et al.  Modeling working memory: An interference model of complex span , 2012, Psychonomic bulletin & review.

[14]  W. Ma,et al.  Changing concepts of working memory , 2014, Nature Neuroscience.

[15]  Adam C. Riggall,et al.  Distributed Patterns of Activity in Sensory Cortex Reflect the Precision of Multiple Items Maintained in Visual Short-Term Memory , 2013, The Journal of Neuroscience.

[16]  John T. Serences,et al.  Reconstructions of Information in Visual Spatial Working Memory Degrade with Memory Load , 2014, Current Biology.

[17]  Flora Bouchacourt,et al.  A Flexible Model of Working Memory , 2019, Neuron.

[18]  Paul M Bays,et al.  Temporal dynamics of encoding, storage, and reallocation of visual working memory. , 2011, Journal of vision.

[19]  Klaus Oberauer,et al.  Visual working memory declines when more features must be remembered for each object , 2013, Memory & cognition.

[20]  A. Pouget,et al.  Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability , 2012, Neuron.

[21]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[22]  J. Palmer Attentional limits on the perception and memory of visual information. , 1990, Journal of experimental psychology. Human perception and performance.

[23]  G. Orban,et al.  The response variability of striate cortical neurons in the behaving monkey , 2004, Experimental Brain Research.

[24]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[25]  Dominic Standage,et al.  Slot-like capacity and resource-like coding in a neural model of multiple-item working memory , 2017, bioRxiv.

[26]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  Eero P. Simoncelli,et al.  Efficient Sensory Encoding and Bayesian Inference with Heterogeneous Neural Populations , 2014, Neural Computation.

[28]  Xue-Xin Wei,et al.  A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts , 2015, Nature Neuroscience.

[29]  Thomas L. Griffiths,et al.  One and Done? Optimal Decisions From Very Few Samples , 2014, Cogn. Sci..

[30]  Edward Awh,et al.  Clear evidence for item limits in visual working memory , 2017, Cognitive Psychology.

[31]  Paul M Bays,et al.  The precision of visual working memory is set by allocation of a shared resource. , 2009, Journal of vision.

[32]  Robert Taylor,et al.  Stochastic sampling provides a unifying account of visual working memory limits , 2020, Proceedings of the National Academy of Sciences.

[33]  Karl R Gegenfurtner,et al.  Dynamics of oculomotor direction discrimination. , 2012, Journal of vision.

[34]  S. Luck,et al.  Discrete fixed-resolution representations in visual working memory , 2008, Nature.

[35]  Xiao-Jing Wang,et al.  From Distributed Resources to Limited Slots in Multiple-Item Working Memory: A Spiking Network Model with Normalization , 2012, The Journal of Neuroscience.

[36]  Philip L. Smith The Poisson shot noise model of visual short-term memory and choice response time: Normalized coding by neural population size , 2015 .

[37]  Eero P. Simoncelli,et al.  Partitioning neuronal variability , 2014, Nature Neuroscience.

[38]  P. Cavanagh,et al.  The Capacity of Visual Short-Term Memory is Set Both by Visual Information Load and by Number of Objects , 2004, Psychological science.

[39]  Flora Bouchacourt,et al.  A flexible model of working memory , 2018 .

[40]  Paul M. Bays,et al.  A signature of neural coding at human perceptual limits , 2016, bioRxiv.

[41]  Edward Awh,et al.  Item-specific delay activity demonstrates concurrent storage of multiple active neural representations in working memory , 2019, PLoS biology.

[42]  H. Sompolinsky,et al.  Population coding in neuronal systems with correlated noise. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[43]  Wei Ji Ma,et al.  Strategic allocation of working memory resource , 2018, bioRxiv.

[44]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[45]  Paul M. Bays Correspondence between population coding and psychophysical scaling models of working memory , 2019 .

[46]  M. Shadlen,et al.  Decision Making and Sequential Sampling from Memory , 2016, Neuron.

[47]  R. Zemel,et al.  Inference and computation with population codes. , 2003, Annual review of neuroscience.

[48]  Jeffrey S. Johnson,et al.  A Dynamic Neural Field Model of Visual Working Memory and Change Detection , 2009, Psychological science.

[49]  Alexander S. Ecker,et al.  The Effect of Noise Correlations in Populations of Diversely Tuned Neurons , 2011, The Journal of Neuroscience.

[50]  Edward K. Vogel,et al.  The capacity of visual working memory for features and conjunctions , 1997, Nature.

[51]  Robert Taylor,et al.  Efficient Coding in Visual Working Memory Accounts for Stimulus-Specific Variations in Recall , 2018, The Journal of Neuroscience.

[52]  Françoise Viénot,et al.  A study on chromatic contrast regulation , 2004 .

[53]  Paul M Bays,et al.  Noise in Neural Populations Accounts for Errors in Working Memory , 2014, The Journal of Neuroscience.

[54]  Zuzanna Klyszejko,et al.  Attentional priority determines working memory precision , 2014, Vision Research.

[55]  A. Pouget,et al.  Information-limiting correlations , 2014, Nature Neuroscience.

[56]  James H Elder,et al.  Cue dynamics underlying rapid detection of animals in natural scenes. , 2009, Journal of vision.

[57]  Paul M Bays,et al.  Drift in Neural Population Activity Causes Working Memory to Deteriorate Over Time , 2018, The Journal of Neuroscience.

[58]  Haim Sompolinsky,et al.  Erratum: Population coding in neuronal systems with correlated noise [Phys. Rev. E 64, 051904 (2001)] , 2002 .

[59]  Klaus Oberauer,et al.  An Interference Model of Visual Working Memory , 2017, Psychological review.

[60]  Paul M. Bays,et al.  Theory of neural coding predicts an upper bound on estimates of memory variability , 2019 .

[61]  Aspen H. Yoo,et al.  Fechner’s Law in Metacognition: A Quantitative Model of Visual Working Memory Confidence , 2017, Psychological review.

[62]  Gerald Westheimer,et al.  Quantifying target conspicuity in contextual modulation by visual search. , 2011, Journal of vision.

[63]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[64]  Richard L. Lewis,et al.  The mind and brain of short-term memory. , 2008, Annual review of psychology.

[65]  József Fiser,et al.  Neural Variability and Sampling-Based Probabilistic Representations in the Visual Cortex , 2016, Neuron.

[66]  Simon D Lilburn,et al.  An information capacity limitation of visual short-term memory. , 2014, Journal of experimental psychology. Human perception and performance.

[67]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[68]  George A. Alvarez,et al.  Variability in the quality of visual working memory , 2012, Nature Communications.

[69]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[70]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[71]  J Miller,et al.  Attentional effects on concurrent psychophysical discriminations: Investigations of a sample-size model , 1994, Perception & psychophysics.