Hebbian Wiring Plasticity Generates Efficient Network Structures for Robust Inference with Synaptic Weight Plasticity

In the adult mammalian cortex, a small fraction of spines are created and eliminated every day, and the resultant synaptic connection structure is highly nonrandom, even in local circuits. However, it remains unknown whether a particular synaptic connection structure is functionally advantageous in local circuits, and why creation and elimination of synaptic connections is necessary in addition to rich synaptic weight plasticity. To answer these questions, we studied an inference task model through theoretical and numerical analyses. We demonstrate that a robustly beneficial network structure naturally emerges by combining Hebbian-type synaptic weight plasticity and wiring plasticity. Especially in a sparsely connected network, wiring plasticity achieves reliable computation by enabling efficient information transmission. Furthermore, the proposed rule reproduces experimental observed correlation between spine dynamics and task performance.

[1]  S. Denéve,et al.  Neural processing as causal inference , 2011, Current Opinion in Neurobiology.

[2]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[3]  W. Gan,et al.  Development of Long-Term Dendritic Spine Stability in Diverse Regions of Cerebral Cortex , 2005, Neuron.

[4]  C. Petersen,et al.  The Excitatory Neuronal Network of the C2 Barrel Column in Mouse Primary Somatosensory Cortex , 2009, Neuron.

[5]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[6]  K. Svoboda,et al.  Experience-dependent structural synaptic plasticity in the mammalian brain , 2009, Nature Reviews Neuroscience.

[7]  M. J. Friedlander,et al.  The time course and amplitude of EPSPs evoked at synapses between pairs of CA3/CA1 neurons in the hippocampal slice , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Michele Pignatelli,et al.  Engram cells retain memory under retrograde amnesia , 2015, Science.

[9]  Wulfram Gerstner,et al.  Spike-timing dependent plasticity , 2010, Scholarpedia.

[10]  Stefan Habenschuss,et al.  Emergence of Optimal Decoding of Population Codes Through STDP , 2013, Neural Computation.

[11]  K. Steinbuch,et al.  Information and Efficiency , 2019, Public Finance and Public Policy.

[12]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[13]  ChechikGal,et al.  Synaptic pruning in development , 1998 .

[14]  A. Faisal,et al.  Noise in the nervous system , 2008, Nature Reviews Neuroscience.

[15]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[16]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[17]  Ziv Bar-Joseph,et al.  Decreasing-Rate Pruning Optimizes the Construction of Efficient and Robust Distributed Networks , 2015, PLoS Comput. Biol..

[18]  MaassWolfgang,et al.  Real-time computing without stable states , 2002 .

[19]  T. Bliss,et al.  A synaptic model of memory: long-term potentiation in the hippocampus , 1993, Nature.

[20]  H. Sompolinsky,et al.  Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. , 2012, Annual review of neuroscience.

[21]  W. Gan,et al.  Sleep promotes branch-specific formation of dendritic spines after learning , 2014, Science.

[22]  G. Buzsáki,et al.  The log-dynamic brain: how skewed distributions affect network operations , 2014, Nature Reviews Neuroscience.

[23]  Tobias C. Potjans,et al.  The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model , 2012, Cerebral cortex.

[24]  Isaac Meilijson,et al.  Synaptic Pruning in Development: A Computational Account , 1998, Neural Computation.

[25]  Bartlett W. Mel,et al.  Cortical rewiring and information storage , 2004, Nature.

[26]  Tomoki Fukai,et al.  Associative memory model with long-tail-distributed Hebbian synaptic connections , 2013, Front. Comput. Neurosci..

[27]  Florentin Wörgötter,et al.  The Formation of Multi-synaptic Connections by the Interaction of Synaptic and Structural Plasticity and Their Functional Consequences , 2014, BMC Neuroscience.

[28]  Kaspar Podgorski,et al.  Rapid Hebbian axonal remodeling mediated by visual stimulation , 2014, Science.

[29]  W. Gan,et al.  Stably maintained dendritic spines are associated with lifelong memories , 2009, Nature.

[30]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[31]  T. Sejnowski,et al.  Nanoconnectomic upper bound on the variability of synaptic plasticity , 2015, eLife.

[32]  Yoshikazu Isomura,et al.  Two distinct layer-specific dynamics of cortical ensembles during learning of a motor task , 2014, Nature Neuroscience.

[33]  J. Nadal,et al.  Optimal Information Storage and the Distribution of Synaptic Weights Perceptron versus Purkinje Cell , 2004, Neuron.

[34]  Daeyeol Lee,et al.  Role of rodent secondary motor cortex in value-based action selection , 2011, Nature Neuroscience.

[35]  Christian Tetzlaff,et al.  The formation of multi-synaptic connections by the interaction of synaptic and structural plasticity and their functional consequences , 2014, BMC Neuroscience.

[36]  E Salinas,et al.  Conversion of Sensory Signals into Motor Commands in Primary Motor Cortex , 1998, The Journal of Neuroscience.

[37]  Thomas K. Berger,et al.  A synaptic organizing principle for cortical neuronal groups , 2011, Proceedings of the National Academy of Sciences.

[38]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[39]  G. Ellis‐Davies,et al.  Structural basis of long-term potentiation in single dendritic spines , 2004, Nature.

[40]  Bartlett W. Mel,et al.  Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural Tissue , 2001, Neuron.

[41]  M. G. Faulkner,et al.  Engram cells retain memory under retrograde amnesia , 2015, Science.

[42]  D. Chklovskii,et al.  Wiring optimization can relate neuronal structure and function. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[43]  E. Callaway,et al.  Excitatory cortical neurons form fine-scale functional networks , 2005, Nature.

[44]  H. Sompolinsky,et al.  Sparseness and Expansion in Sensory Representations , 2014, Neuron.

[45]  K. Svoboda,et al.  Spine growth precedes synapse formation in the adult neocortex in vivo , 2006, Nature Neuroscience.

[46]  D. Feldman.  Synaptic mechanisms for plasticity in neocortex. , 2009, Annual review of neuroscience.

[47]  Günther Palm,et al.  Memory Capacities for Synaptic and Structural Plasticity G ¨ Unther Palm , 2022 .

[48]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[49]  E. Miller,et al.  A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales , 2007, Neuron.

[50]  Moritz Helias,et al.  Spike-Timing Dependence of Structural Plasticity Explains Cooperative Synapse Formation in the Neocortex , 2012, PLoS Comput. Biol..

[51]  D. Chklovskii,et al.  Geometry and Structural Plasticity of Synaptic Connectivity , 2002, Neuron.

[52]  L. Christophorou.  Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[53]  Lav R. Varshney,et al.  Optimal Information Storage in Noisy Synapses under Resource Constraints , 2006, Neuron.

[54]  Mark C. W. van Rossum,et al.  Energy Efficient Sparse Connectivity from Imbalanced Synaptic Plasticity Rules , 2015, PLoS Comput. Biol..

[55]  G. Shepherd,et al.  Transient and Persistent Dendritic Spines in the Neocortex In Vivo , 2005, Neuron.

[56]  S. Nelson,et al.  Homeostatic plasticity in the developing nervous system , 2004, Nature Reviews Neuroscience.

[57]  J. Simon Wiegert,et al.  Long-term depression triggers the selective elimination of weakly integrated synapses , 2013, Proceedings of the National Academy of Sciences.

[58]  Christos Dimitrakakis,et al.  Network Self-Organization Explains the Statistics and Dynamics of Synaptic Connection Strengths in Cortex , 2013, PLoS Comput. Biol..

[59]  N. Matsuki,et al.  Interpyramid spike transmission stabilizes the sparseness of recurrent network activity. , 2013, Cerebral cortex.

[60]  Masaharu Ogawa,et al.  BTBD3 Controls Dendrite Orientation Toward Active Axons in Mammalian Neocortex , 2013, Science.

[61]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[62]  H. Kasai,et al.  Principles of Long-Term Dynamics of Dendritic Spines , 2008, The Journal of Neuroscience.

[63]  L. Pinneo.  On noise in the nervous system. , 1966, Psychological review.

[64]  Willie F. Tobin,et al.  Rapid formation and selective stabilization of synapses for enduring motor memories , 2009, Nature.

[65]  P. Caroni,et al.  Structural plasticity upon learning: regulation and functions , 2012, Nature Reviews Neuroscience.

[66]  Karl J. Friston,et al.  Information and Efficiency in the Nervous System—A Synthesis , 2013, PLoS Comput. Biol..

[67]  C. Clopath,et al.  The emergence of functional microcircuits in visual cortex , 2013, Nature.

[68]  Sho Yagishita,et al.  Learning rules and persistence of dendritic spines , 2010, The European journal of neuroscience.