Hebbian Wiring Plasticity Generates Efficient Network Structures for Robust Inference with Synaptic Weight Plasticity

In the adult mammalian cortex, a small fraction of spines are created and eliminated every day, and the resultant synaptic connection structure is highly nonrandom, even in local circuits. However, it remains unknown whether a particular synaptic connection structure is functionally advantageous in local circuits, and why creation and elimination of synaptic connections is necessary in addition to rich synaptic weight plasticity. To answer these questions, we studied an inference task model through theoretical and numerical analyses. We demonstrate that a robustly beneficial network structure naturally emerges by combining Hebbian-type synaptic weight plasticity and wiring plasticity. Especially in a sparsely connected network, wiring plasticity achieves reliable computation by enabling efficient information transmission. Furthermore, the proposed rule reproduces experimental observed correlation between spine dynamics and task performance.

[1]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[2]  P. Caroni,et al.  Structural plasticity upon learning: regulation and functions , 2012, Nature Reviews Neuroscience.

[3]  Willie F. Tobin,et al.  Rapid formation and selective stabilization of synapses for enduring motor memories , 2009, Nature.

[4]  Karl J. Friston,et al.  Information and Efficiency in the Nervous System—A Synthesis , 2013, PLoS Comput. Biol..

[5]  Daeyeol Lee,et al.  Role of rodent secondary motor cortex in value-based action selection , 2011, Nature Neuroscience.

[6]  Kaspar Podgorski,et al.  Rapid Hebbian axonal remodeling mediated by visual stimulation , 2014, Science.

[7]  Bartlett W. Mel,et al.  Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural Tissue , 2001, Neuron.

[8]  K. Svoboda,et al.  Spine growth precedes synapse formation in the adult neocortex in vivo , 2006, Nature Neuroscience.

[9]  Tomoki Fukai,et al.  Associative memory model with long-tail-distributed Hebbian synaptic connections , 2013, Front. Comput. Neurosci..

[10]  W. Gan,et al.  Development of Long-Term Dendritic Spine Stability in Diverse Regions of Cerebral Cortex , 2005, Neuron.

[11]  Bartlett W. Mel,et al.  Cortical rewiring and information storage , 2004, Nature.

[12]  J. Simon Wiegert,et al.  Long-term depression triggers the selective elimination of weakly integrated synapses , 2013, Proceedings of the National Academy of Sciences.

[13]  ChechikGal,et al.  Synaptic pruning in development , 1998 .

[14]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[15]  W. Gan,et al.  Stably maintained dendritic spines are associated with lifelong memories , 2009, Nature.

[16]  Yoshikazu Isomura,et al.  Two distinct layer-specific dynamics of cortical ensembles during learning of a motor task , 2014, Nature Neuroscience.

[17]  J. Nadal,et al.  Optimal Information Storage and the Distribution of Synaptic Weights Perceptron versus Purkinje Cell , 2004, Neuron.

[18]  T. Bliss,et al.  A synaptic model of memory: long-term potentiation in the hippocampus , 1993, Nature.

[19]  Isaac Meilijson,et al.  Synaptic Pruning in Development: A Computational Account , 1998, Neural Computation.

[20]  Sho Yagishita,et al.  Learning rules and persistence of dendritic spines , 2010, The European journal of neuroscience.

[21]  W. Gan,et al.  Sleep promotes branch-specific formation of dendritic spines after learning , 2014, Science.

[22]  Stefan Habenschuss,et al.  Emergence of Optimal Decoding of Population Codes Through STDP , 2013, Neural Computation.

[23]  E Salinas,et al.  Conversion of Sensory Signals into Motor Commands in Primary Motor Cortex , 1998, The Journal of Neuroscience.

[24]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[25]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[26]  C. Clopath,et al.  The emergence of functional microcircuits in visual cortex , 2013, Nature.

[27]  T. Sejnowski,et al.  Nanoconnectomic upper bound on the variability of synaptic plasticity , 2015, eLife.

[28]  Masaharu Ogawa,et al.  BTBD3 Controls Dendrite Orientation Toward Active Axons in Mammalian Neocortex , 2013, Science.

[29]  N. Matsuki,et al.  Interpyramid spike transmission stabilizes the sparseness of recurrent network activity. , 2013, Cerebral cortex.

[30]  H. Sompolinsky,et al.  Sparseness and Expansion in Sensory Representations , 2014, Neuron.

[31]  Mark C. W. van Rossum,et al.  Energy Efficient Sparse Connectivity from Imbalanced Synaptic Plasticity Rules , 2015, PLoS Comput. Biol..

[32]  H. Kasai,et al.  Principles of Long-Term Dynamics of Dendritic Spines , 2008, The Journal of Neuroscience.

[33]  Christian Tetzlaff,et al.  The formation of multi-synaptic connections by the interaction of synaptic and structural plasticity and their functional consequences , 2014, BMC Neuroscience.

[34]  Thomas K. Berger,et al.  A synaptic organizing principle for cortical neuronal groups , 2011, Proceedings of the National Academy of Sciences.

[35]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[36]  Ziv Bar-Joseph,et al.  Decreasing-Rate Pruning Optimizes the Construction of Efficient and Robust Distributed Networks , 2015, PLoS Comput. Biol..

[37]  M. J. Friedlander,et al.  The time course and amplitude of EPSPs evoked at synapses between pairs of CA3/CA1 neurons in the hippocampal slice , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[38]  G. Buzsáki,et al.  The log-dynamic brain: how skewed distributions affect network operations , 2014, Nature Reviews Neuroscience.

[39]  Tobias C. Potjans,et al.  The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model , 2012, Cerebral cortex.

[40]  D. Feldman.  Synaptic mechanisms for plasticity in neocortex. , 2009, Annual review of neuroscience.

[41]  S. Nelson,et al.  Homeostatic plasticity in the developing nervous system , 2004, Nature Reviews Neuroscience.

[42]  K. Svoboda,et al.  Experience-dependent structural synaptic plasticity in the mammalian brain , 2009, Nature Reviews Neuroscience.

[43]  Florentin Wörgötter,et al.  The Formation of Multi-synaptic Connections by the Interaction of Synaptic and Structural Plasticity and Their Functional Consequences , 2014, BMC Neuroscience.

[44]  M. G. Faulkner,et al.  Engram cells retain memory under retrograde amnesia , 2015, Science.

[45]  H. Sompolinsky,et al.  Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. , 2012, Annual review of neuroscience.

[46]  E. Miller,et al.  A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales , 2007, Neuron.

[47]  Moritz Helias,et al.  Spike-Timing Dependence of Structural Plasticity Explains Cooperative Synapse Formation in the Neocortex , 2012, PLoS Comput. Biol..

[48]  Bruno A. Olshausen,et al.  Book Review , 2003, Journal of Cognitive Neuroscience.

[49]  E. Callaway,et al.  Excitatory cortical neurons form fine-scale functional networks , 2005, Nature.

[50]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[51]  Michele Pignatelli,et al.  Engram cells retain memory under retrograde amnesia , 2015, Science.

[52]  D. Chklovskii,et al.  Geometry and Structural Plasticity of Synaptic Connectivity , 2002, Neuron.

[53]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[54]  Günther Palm,et al.  Memory Capacities for Synaptic and Structural Plasticity G ¨ Unther Palm , 2022 .

[55]  Christos Dimitrakakis,et al.  Network Self-Organization Explains the Statistics and Dynamics of Synaptic Connection Strengths in Cortex , 2013, PLoS Comput. Biol..

[56]  G. Shepherd,et al.  Transient and Persistent Dendritic Spines in the Neocortex In Vivo , 2005, Neuron.

[57]  G. Ellis‐Davies,et al.  Structural basis of long-term potentiation in single dendritic spines , 2004, Nature.

[58]  C. Petersen,et al.  The Excitatory Neuronal Network of the C2 Barrel Column in Mouse Primary Somatosensory Cortex , 2009, Neuron.

[59]  L. Pinneo.  On noise in the nervous system. , 1966, Psychological review.

[60]  Lav R. Varshney,et al.  Optimal Information Storage in Noisy Synapses under Resource Constraints , 2006, Neuron.

[61]  D. Chklovskii,et al.  Wiring optimization can relate neuronal structure and function. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[62]  S. Denéve,et al.  Neural processing as causal inference , 2011, Current Opinion in Neurobiology.