Maximum Memory Capacity on Neural Networks with Short-Term Synaptic Depression and Facilitation

In this work, we study, analytically and employing Monte Carlo simulations, the influence of the competition between several activity-dependent synaptic processes, such as short-term synaptic facilitation and depression, on the maximum memory storage capacity in a neural network. In contrast to the case of synaptic depression, which drastically reduces the capacity of the network to store and retrieve static activity patterns, synaptic facilitation enhances the storage capacity in different contexts. In particular, we found optimal values of the relevant synaptic parameters (such as the neurotransmitter release probability or the characteristic facilitation time constant) for which the storage capacity can be maximal and similar to the one obtained with static synapses, that is, without activity-dependent processes. We conclude that depressing synapses with a certain level of facilitation allow recovering the good retrieval properties of networks with static synapses while maintaining the nonlinear characteristics of dynamic synapses, convenient for information processing and coding.

[1]  H. Markram,et al.  Differential signaling via the same axon of neocortical pyramidal neurons. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[2]  L. Abbott,et al.  Synaptic computation , 2004, Nature.

[3]  N. Parga,et al.  Short-term Synaptic Depression Causes a Non-monotonic Response to Correlated Stimuli , 2022 .

[4]  Masato Okada,et al.  Statistical mechanics of attractor neural network models with synaptic depression , 2009 .

[5]  Hilbert J. Kappen,et al.  Associative Memory with Dynamic Synapses , 2002, Neural Computation.

[6]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[7]  S. Amari,et al.  Characteristics of Random Nets of Analog Neuron-Like Elements , 1972, IEEE Trans. Syst. Man Cybern..

[8]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[9]  Shiino,et al.  Self-consistent signal-to-noise analysis of the statistical behavior of analog neural networks and enhancement of the storage capacity. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[10]  Daniel J. Amit,et al.  Mean-field analysis of selective persistent activity in presence of short-term synaptic depression , 2006, Journal of Computational Neuroscience.

[11]  Joaquín J. Torres,et al.  Influence of topology on the performance of a neural network , 2004, Neurocomputing.

[12]  R. Bertram,et al.  Single-domain/bound calcium hypothesis of transmitter release and facilitation. , 1996, Journal of neurophysiology.

[13]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[14]  D. Amit,et al.  Quantitative study of attractor neural networks retrieving at low spike rates: II. Low-rate retrieval in symmetric networks , 1991 .

[15]  Masato Okada,et al.  Retrieval Property of Attractor Network with Synaptic Depression(General) , 2007 .

[16]  L. Abbott,et al.  Limits on the memory storage capacity of bounded synapses , 2007, Nature Neuroscience.

[17]  R. Zucker,et al.  Residual Ca2 + and short-term synaptic plasticity , 1994, Nature.

[18]  Sompolinsky,et al.  Information storage in neural networks with low levels of activity. , 1987, Physical review. A, General physics.

[19]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[20]  P. Peretto.  An introduction to the modeling of neural networks , 1992 .

[21]  C. Gray,et al.  Dynamic spike threshold reveals a mechanism for synaptic coincidence detection in cortical neurons in vivo. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Misha Tsodyks,et al.  The Emergence of Up and Down States in Cortical Networks , 2006, PLoS Comput. Biol..

[23]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[24]  T. Geisel,et al.  Pattern storage and processing in attractor networks with short-time synaptic dynamics , 2002, Network.

[25]  Misha Tsodyks,et al.  Persistent Activity in Neural Networks with Dynamic Synapses , 2007, PLoS Comput. Biol..

[26]  Horn,et al.  Neural networks with dynamical thresholds. , 1989, Physical review. A, General physics.

[27]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .

[28]  Lovorka Pantic,et al.  Storage capacity of attractor neural networks with depressing synapses. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[30]  Hilbert J. Kappen,et al.  Competition Between Synaptic Depression and Facilitation in Attractor Neural Networks , 2006, Neural Computation.

[31]  Masato Okada,et al.  Instabilities in Associative Memory Model with Synaptic Depression and Switching Phenomena among Attractors , 2010 .

[32]  Thomas K. Berger,et al.  Heterogeneity in the pyramidal network of the medial prefrontal cortex , 2006, Nature Neuroscience.

[33]  Daniel J. Amit,et al.  Quantitative Study of Attractor Neural Network Retrieving at Low Spike Rates: I , 1991 .

[34]  L. Abbott,et al.  Synaptic Depression and Cortical Gain Control , 1997, Science.

[35]  C. Stevens,et al.  Heterogeneity of Release Probability, Facilitation, and Depletion at Central Synapses , 1997, Neuron.

[36]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[37]  Michael Menzinger,et al.  Topology and computational performance of attractor neural networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[38]  Joaquín J. Torres,et al.  The role of synaptic facilitation in spike coincidence detection , 2008, Journal of Computational Neuroscience.

[39]  Chunguang Li,et al.  Synchronization of the small-world neuronal network with unreliable synapses , 2010, Physical biology.

[40]  M. Tsodyks,et al.  Synaptic Theory of Working Memory , 2008, Science.