Critical dynamics in associative memory networks

Critical behavior in neural networks is characterized by scale-free avalanche size distributions and can be explained by self-regulatory mechanisms. Theoretical and experimental evidence indicates that information storage capacity reaches its maximum in the critical regime. We study the effect of structural connectivity formed by Hebbian learning on the criticality of network dynamics. The network only endowed with Hebbian learning does not allow for simultaneous information storage and criticality. However, the critical regime can be stabilized by short-term synaptic dynamics in the form of synaptic depression and facilitation or, alternatively, by homeostatic adaptation of the synaptic weights. We show that a heterogeneous distribution of maximal synaptic strengths does not preclude criticality if the Hebbian learning is alternated with periods of critical dynamics recovery. We discuss the relevance of these findings for the flexibility of memory in aging and with respect to the recent theory of synaptic plasticity.

[1]  J. Michael Herrmann,et al.  Dynamical Synapses Give Rise to a Power-Law Distribution of Neuronal Avalanches , 2005, NIPS.

[2]  J. Michael Herrmann,et al.  Critical dynamics in homeostatic memory networks , 2011 .

[3]  J. M. Herrmann,et al.  Finite-size effects of avalanche dynamics. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  J. M. Herrmann,et al.  Phase transitions towards criticality in a neural system with adaptive interactions. , 2009, Physical review letters.

[5]  R. Otter.  The Multiplicative Process , 1949 .

[6]  Min Lin,et al.  Self-organized criticality in a simple model of neurons based on small-world networks. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[7]  J. Michael Herrmann,et al.  Criticality of avalanche dynamics in adaptive recurrent networks , 2006, Neurocomputing.

[8]  M. Tsodyks.  ASSOCIATIVE MEMORY IN NEURAL NETWORKS WITH THE HEBBIAN LEARNING RULE , 1989 .

[9]  John M. Beggs,et al.  Neuronal Avalanches in Neocortical Circuits , 2003, The Journal of Neuroscience.

[10]  Henry Markram,et al.  Plasticity of Neocortical Synapses Enables Transitions between Rate and Temporal Coding , 1996, ICANN.

[11]  Mark T. Waters,et al.  This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits distribution,andreproductioninanymedium,providedtheoriginalauthorandsourcearecredited.Thislicensedoesnot permit commercial exploitation or the creation of derivative works without sp , 2009 .

[12]  D. Chialvo.  Are our senses critical? , 2006 .

[13]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[14]  Alessandro Treves,et al.  Frontal latching networks: a possible neural basis for infinite recursion , 2005, Cognitive neuropsychology.

[15]  Gilles Laurent,et al.  Transient Dynamics for Neural Processing , 2008, Science.

[16]  Henry Markram,et al.  The "Liquid Computer": A Novel Strategy for Real-Time Computing on Time Series , 2002 .

[17]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[18]  L. F. Abbott,et al.  Analysis of Neuron Models with Dynamically Regulated Conductances , 1993, Neural Computation.

[19]  Daniel J. Amit,et al.  Mean-field analysis of selective persistent activity in presence of short-term synaptic depression , 2006, Journal of Computational Neuroscience.

[20]  John M Beggs,et al.  Critical branching captures activity in living neural networks and maximizes the number of metastable States. , 2005, Physical review letters.

[21]  Tomoki Fukai,et al.  Local cortical circuit model inferred from power-law distributed neuronal avalanches , 2007, Journal of Computational Neuroscience.

[22]  Are age-related cognitive effects caused by optimization? , 2009, BMC Neuroscience.

[23]  M. Tsodyks,et al.  The Enhanced Storage Capacity in Neural Networks with Low Activity Level , 1988 .

[24]  Woodrow L. Shew,et al.  Predicting criticality and dynamic range in complex networks: effects of topology. , 2010, Physical review letters.

[25]  Paolo Del Giudice,et al.  Long and short-term synaptic plasticity and the formation of working memory: A case study , 2001, Neurocomputing.

[26]  Niraj S. Desai,et al.  Activity-dependent scaling of quantal amplitude in neocortical neurons , 1998, Nature.

[27]  Anna Levina,et al.  A Mathematical Approach to Self-Organized Criticality in Neural Networks , 2008 .

[28]  Horn,et al.  Neural networks with dynamical thresholds. , 1989, Physical review. A, General physics.

[29]  Marius Usher,et al.  A neural model of the dynamic activation of memory , 1993, Biological Cybernetics.

[30]  S. Kauffman,et al.  Measures for information propagation in Boolean networks , 2007 .

[31]  J. M. Herrmann,et al.  Dynamical synapses causing self-organized criticality in neural networks , 2007, 0712.1003.

[32]  J. Hopfield,et al.  Earthquake cycles and neural reverberations: Collective oscillations in systems with pulse-coupled threshold elements. , 1995, Physical review letters.

[33]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[35]  Woodrow L. Shew,et al.  The Functional Benefits of Criticality in the Cortex , 2013, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[36]  Adam Prügel-Bennett,et al.  Analysis of synfire chains , 1995 .

[37]  H. Markram,et al.  The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[38]  T. Geisel,et al.  Pattern storage and processing in attractor networks with short-time synaptic dynamics , 2002, Network.