Capacity analysis in multi-state synaptic models: a retrieval probability perspective

We define the memory capacity of networks of binary neurons with finite-state synapses in terms of retrieval probabilities of learned patterns under standard asynchronous dynamics with a predetermined threshold. The threshold is set to control the proportion of non-selective neurons that fire. An optimal inhibition level is chosen to stabilize network behavior. For any local learning rule we provide a computationally efficient and highly accurate approximation to the retrieval probability of a pattern as a function of its age. The method is applied to the sequential models (Fusi and Abbott, Nat Neurosci 10:485–493, 2007) and meta-plasticity models (Fusi et al., Neuron 45(4):599–611, 2005; Leibold and Kempter, Cereb Cortex 18:67–77, 2008). We show that as the number of synaptic states increases, the capacity, as defined here, either plateaus or decreases. In the few cases where multi-state models exceed the capacity of binary synapse models the improvement is small.

[1]  B. McNaughton,et al.  Comparison of spatial and temporal characteristics of neuronal activity in sequential stages of hippocampal processing. , 1990, Progress in brain research.

[2]  Mark C. W. van Rossum,et al.  Optimal Learning Rules for Discrete Synapses , 2008, PLoS Comput. Biol..

[3]  R. Kempter,et al.  Sparseness constrains the prolongation of memory lifetime via synaptic metaplasticity. , 2008, Cerebral cortex.

[4]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[5]  Koichiro Deguchi,et al.  Comparison of spatial and temporal characteristic between reflection-type tactile sensor and human cutaneous sensation , 2009, RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication.

[6]  P. Lennie.  The Cost of Cortical Computation , 2003, Current Biology.

[7]  Daniel J. Amit,et al.  Mean Field and Capacity in Realistic Networks of Spiking Neurons Storing Sparsely Coded Random Memories , 2004, Neural Computation.

[8]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[9]  Yali Amit,et al.  Precise Capacity Analysis in Binary Networks with Multiple Coding Level Inputs , 2010, Neural Computation.

[10]  Y. Miyashita,et al.  Neural representation of visual objects: encoding and top-down activation , 2000, Current Opinion in Neurobiology.

[11]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[12]  Go Uchida,et al.  Nature of Neural Clustering in Inferotemporal Cortex of Macaque Monkey , 2006 .

[13]  Xiao-Jing Wang.  Synaptic reverberation underlying mnemonic persistent activity , 2001, Trends in Neurosciences.

[14]  Daniel J. Amit,et al.  Learning in Neural Networks with Material Synapses , 1994, Neural Computation.

[15]  P. D. Giudice,et al.  Modelling the formation of working memory with networks of integrate-and-fire neurons connected by plastic synapses , 2003, Journal of Physiology-Paris.

[16]  Gianluigi Mongillo,et al.  Selective delay activity in the cortex: phenomena and interpretation. , 2003, Cerebral cortex.

[17]  E T Rolls,et al.  Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. , 1995, Journal of neurophysiology.

[18]  Ned S. Wingreen,et al.  Chemotaxis in Escherichia coli: A Molecular Model for Robust Precise Adaptation , 2007, PLoS Comput. Biol..

[19]  Stefano Fusi,et al.  Long Memory Lifetimes Require Complex Synapses and Limited Sparseness , 2007, Frontiers Comput. Neurosci..

[20]  Jane S. Paulsen.  Memory in the Cerebral Cortex: An Empirical Approach to Neural Networks in the Human and Nonhuman Primate , 1996 .

[21]  Nicolas Brunel,et al.  Dynamics of a recurrent network of spiking neurons before and following learning , 1997 .

[22]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[23]  S. Laughlin,et al.  An Energy Budget for Signaling in the Grey Matter of the Brain , 2001, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[24]  Nicolas Brunel,et al.  Dynamics and plasticity of stimulus-selective persistent activity in cortical network models. , 2003, Cerebral cortex.

[25]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[26]  L. Abbott,et al.  Cascade Models of Synaptically Stored Memories , 2005, Neuron.

[27]  Yali Amit,et al.  Optimizing One-Shot Learning with Binary Synapses , 2008, Neural Computation.

[28]  M. Bear,et al.  Metaplasticity: the plasticity of synaptic plasticity , 1996, Trends in Neurosciences.

[29]  B. McNaughton,et al.  Spatial selectivity of unit activity in the hippocampal granular layer , 1993, Hippocampus.

[30]  Peter Dayan,et al.  Bilinearity, Rules, and Prefrontal Cortex , 2007, Frontiers Comput. Neurosci..

[31]  L. Abbott,et al.  Limits on the memory storage capacity of bounded synapses , 2007, Nature Neuroscience.