Stability Against Fluctuations: Scaling, Bifurcations, and Spontaneous Symmetry Breaking in Stochastic Models of Synaptic Plasticity

In stochastic models of synaptic plasticity based on a random walk, the control of fluctuations is imperative. We have argued that synapses could act as low-pass filters, filtering plasticity induction steps before expressing a step change in synaptic strength. Earlier work showed, in simulation, that such a synaptic filter tames fluctuations very well, leading to patterns of synaptic connectivity that are stable for long periods of time. Here, we approach this problem analytically. We explicitly calculate the lifetime of meta-stable states of synaptic connectivity using a Fokker-Planck formalism in order to understand the dependence of this lifetime on both the plasticity step size and the filtering mechanism. We find that our analytical results agree very well with simulation results, despite having to make two approximations. Our analysis reveals, however, a deeper significance to the filtering mechanism and the plasticity step size. We show that a filter scales the step size into a smaller, effective step size. This scaling suggests that the step size may itself play the role of a temperature parameter, so that a filter cools the dynamics, thereby reducing the influence of fluctuations. Using the master equation, we explicitly demonstrate a bifurcation at a critical step size, confirming this interpretation. At this critical point, spontaneous symmetry breaking occurs in the class of stochastic models of synaptic plasticity that we consider.

[1]  Jeff W. Lichtman,et al.  Principles of neural development , 1985 .

[2]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[3]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[4]  Carson C. Chow,et al.  Calcium time course as a signal for spike-timing-dependent plasticity. , 2005, Journal of neurophysiology.

[5]  Terry Elliott,et al.  Synaptic and Temporal Ensemble Interpretation of Spike-Timing-Dependent Plasticity , 2005, Neural Computation.

[6]  L. Cooper,et al.  A biophysical model of bidirectional synaptic plasticity: Dependence on AMPA and NMDA receptors , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Daniel D. Lee,et al.  Equilibrium properties of temporally asymmetric Hebbian plasticity. , 2000, Physical review letters.

[8]  C. I. Howarth,et al.  Axonal processes and neural plasticity.I: Ocular dominance columns. , 1996, Cerebral cortex.

[9]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[10]  Terry Elliott,et al.  The Mean Time to Express Synaptic Plasticity in Integrate-and-Express, Stochastic Models of Synaptic Plasticity Induction , 2011, Neural Computation.

[11]  Armando Bazzani,et al.  Toward a microscopic model of bidirectional synaptic plasticity , 2009, Proceedings of the National Academy of Sciences.

[12]  Terry Elliott,et al.  Taming Fluctuations in a Stochastic Model of Spike-Timing-Dependent Plasticity , 2009, Neural Computation.

[13]  R. Malenka,et al.  Coordinated Changes in Dendritic Arborization and Synaptic Strength during Neural Circuit Development , 2009, Neuron.

[14]  Terry Elliott,et al.  The Rise and Fall of Memory in a Model of Synaptic Integration , 2012, Neural Computation.

[15]  Karel Svoboda,et al.  Activity-Dependent Plasticity of the NMDA-Receptor Fractional Ca2+ Current , 2007, Neuron.

[16]  W. Senn,et al.  Convergence of stochastic learning in perceptrons with binary synapses. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  Alan Fine,et al.  Expression of Long-Term Plasticity at Individual Synapses in Hippocampus Is Graded, Bidirectional, and Mainly Presynaptic: Optical Quantal Analysis , 2009, Neuron.

[18]  J. Hopfield,et al.  All-or-none potentiation at CA3-CA1 synapses. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[20]  A. Fine,et al.  Are binary synapses superior to graded weight representations in stochastic attractor networks? , 2009, Cognitive Neurodynamics.

[21]  Terry Elliott,et al.  Discrete States of Synaptic Strength in a Stochastic Model of Spike-Timing-Dependent Plasticity , 2010, Neural Computation.

[22]  N. Swindale.  The development of topography in the visual cortex: a review of models. , 1996, Network.

[23]  David B. Grayden,et al.  Spike-Timing-Dependent Plasticity: The Relationship to Rate-Based Learning for Models with Weight Dynamics Determined by a Stable Fixed Point , 2004, Neural Computation.

[24]  Arjen van Ooyen,et al.  Competition in the development of nerve connections: a review of models , 2001 .

[25]  Bartlett W. Mel,et al.  Cortical rewiring and information storage , 2004, Nature.

[26]  Nicolas Brunel,et al.  STDP in a Bistable Synapse Model Based on CaMKII and Associated Signaling Pathways , 2007, PLoS Comput. Biol..

[27]  Terry Elliott,et al.  A Non-Markovian Random Walk Underlies a Stochastic Model of Spike-Timing-Dependent Plasticity , 2010, Neural Computation.

[28]  S. Wang,et al.  Dissection of bidirectional synaptic plasticity into saturable unidirectional processes. , 2005, Journal of neurophysiology.

[29]  Mark C. W. van Rossum,et al.  Stable Hebbian Learning from Spike Timing-Dependent Plasticity , 2000, The Journal of Neuroscience.

[30]  L. Cooper,et al.  A unified model of NMDA receptor-dependent bidirectional synaptic plasticity , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[31]  Yali Amit,et al.  Optimizing One-Shot Learning with Binary Synapses , 2008, Neural Computation.

[32]  Sachin S Talathi,et al.  Synaptic plasticity with discrete state synapses. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[33]  R. Kempter,et al.  Sparseness constrains the prolongation of memory lifetime via synaptic metaplasticity. , 2008, Cerebral cortex.

[34]  J. Montgomery,et al.  Discrete synaptic states define a major mechanism of synapse plasticity , 2004, Trends in Neurosciences.

[35]  Terry Elliott,et al.  Stable Competitive Dynamics Emerge from Multispike Interactions in a Stochastic Model of Spike-Timing-Dependent Plasticity , 2006, Neural Computation.

[36]  J. Kao,et al.  Long-term potentiation of exogenous glutamate responses at single dendritic spines. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[37]  S. Wang,et al.  Malleability of Spike-Timing-Dependent Plasticity at the CA3–CA1 Synapse , 2006, The Journal of Neuroscience.

[38]  L. Abbott,et al.  Cascade Models of Synaptically Stored Memories , 2005, Neuron.

[39]  J. Montgomery,et al.  State-Dependent Heterogeneity in Synaptic Depression between Pyramidal Cell Pairs , 2002, Neuron.

[40]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[41]  Karel Svoboda,et al.  Plasticity of calcium channels in dendritic spines , 2003, Nature Neuroscience.

[42]  Terry Elliott,et al.  Cross-Talk Induces Bifurcations in Nonlinear Models of Synaptic Plasticity , 2012, Neural Computation.

[43]  Geoffrey E. Hinton,et al.  OPTIMAL PERCEPTUAL INFERENCE , 1983 .

[44]  S. Wang,et al.  Graded bidirectional synaptic plasticity is composed of switch-like unitary events. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[45]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[46]  Walter Senn,et al.  Eluding oblivion with smart stochastic selection of synaptic updates. , 2006, Chaos.

[47]  Henry Markram,et al.  An Algorithm for Modifying Neurotransmitter Release Probability Based on Pre- and Postsynaptic Spike Timing , 2001, Neural Computation.

[48]  Terry Elliott,et al.  Temporal Dynamics of Rate-Based Synaptic Plasticity Rules in a Stochastic Model of Spike-Timing-Dependent Plasticity , 2008, Neural Computation.

[49]  Walter Senn,et al.  Beyond spike timing: the role of nonlinear plasticity and unreliable synapses , 2002, Biological Cybernetics.

[50]  Nigel Shadbolt,et al.  Axonal Processes and Neural Plasticity: A Reply , 1998, Neural Computation.