Structural Plasticity on the SpiNNaker Many-Core Neuromorphic System

The structural organization of cortical areas is not random, with topographic maps commonplace in sensory processing centers. This topographical organization allows optimal wiring between neurons, multimodal sensory integration, and performs input dimensionality reduction. In this work, a model of topographic map formation is implemented on the SpiNNaker neuromorphic platform, running in realtime using point neurons, and making use of both synaptic rewiring and spike-timing dependent plasticity (STDP). In agreement with Bamford et al. (2010), we demonstrate that synaptic rewiring refines an initially rough topographic map over and beyond the ability of STDP, and that input selectivity learnt through STDP is embedded into the network connectivity through rewiring. Moreover, we show the presented model can be used to generate topographic maps between layers of neurons with minimal initial connectivity, and stabilize mappings which would otherwise be unstable through the inclusion of lateral inhibition.

[1]  Marco Tomassini,et al.  Dynamics of pruning in simulated large-scale spiking neural networks. , 2005, Bio Systems.

[2]  Henning Sprekeler,et al.  Inhibitory Plasticity Balances Excitation and Inhibition in Sensory Pathways and Memory Networks , 2011, Science.

[3]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[4]  Willie F. Tobin,et al.  Rapid formation and selective stabilization of synapses for enduring motor memories , 2009, Nature.

[5]  Santiago Ramón y Cajal,et al.  Degeneration & regeneration of the nervous system , 1928 .

[6]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[7]  Steve B. Furber,et al.  Benchmarking Spike-Based Visual Recognition: A Dataset and Evaluation , 2016, Front. Neurosci..

[8]  R. Joseph,et al.  T-cell signaling regulated by the Tec family kinase, Itk. , 2010, Cold Spring Harbor perspectives in biology.

[9]  John H Freeman,et al.  Synapse formation is associated with memory storage in the cerebellum , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Subutai Ahmad,et al.  Why Neurons Have Thousands of Synapses, a Theory of Sequence Memory in Neocortex , 2015, Front. Neural Circuits.

[11]  Bartlett W. Mel,et al.  Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural Tissue , 2001, Neuron.

[12]  K. Svoboda,et al.  Experience-dependent structural synaptic plasticity in the mammalian brain , 2009, Nature Reviews Neuroscience.

[13]  Felix Wolf,et al.  The Rewiring Brain: A Computational Approach to Structural Plasticity in the Adult Brain , 2017 .

[14]  Steve B. Furber,et al.  Understanding the interconnection network of SpiNNaker , 2009, ICS.

[15]  Rüdiger Dillmann,et al.  Neuromorphic Stereo Vision: A Survey of Bio-Inspired Sensors and Algorithms , 2019, Front. Neurorobot..

[16]  Richard George,et al.  Structural Plasticity Denoises Responses and Improves Learning Speed , 2016, Front. Comput. Neurosci..

[17]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[18]  Abigail Morrison,et al.  Automatic Generation of Connectivity for Large-Scale Neuronal Network Models through Structural Plasticity , 2016, Front. Neuroanat..

[19]  David R. Colman,et al.  Molecules, maps and synapse specificity , 2001, Nature Reviews Neuroscience.

[20]  Marc-André Hermanns,et al.  A Scalable Algorithm for Simulating the Structural Plasticity of the Brain , 2016, 2016 28th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD).

[21]  Matthew Cook,et al.  Efficient implementation of STDP rules on SpiNNaker neuromorphic hardware , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[22]  Joseph E LeDoux,et al.  Structural plasticity and memory , 2004, Nature Reviews Neuroscience.

[23]  H. Markram,et al.  Spontaneous and evoked synaptic rewiring in the neonatal neocortex , 2006, Proceedings of the National Academy of Sciences.

[24]  D. Chklovskii,et al.  Maps in the brain: what can we learn from them? , 2004, Annual review of neuroscience.

[25]  N. Shadbolt,et al.  A Neurotrophic Model of the Development of the Retinogeniculocortical Pathway Induced by Spontaneous Retinal Waves , 1999, The Journal of Neuroscience.

[26]  J. Morrison,et al.  The addicted synapse: mechanisms of synaptic and structural plasticity in nucleus accumbens , 2010, Trends in Neurosciences.

[27]  G. Shepherd,et al.  Transient and Persistent Dendritic Spines in the Neocortex In Vivo , 2005, Neuron.

[28]  Steve B. Furber,et al.  Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System , 2018, Front. Neurosci..

[29]  Alexander Peyser,et al.  NEST 2.10.0 , 2015 .

[30]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[31]  Richard George,et al.  Activity dependent structural plasticity in neuromorphic systems , 2017, 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[32]  Markus Butz,et al.  A Simple Rule for Dendritic Spine and Axonal Bouton Formation Can Account for Cortical Reorganization after Focal Retinal Lesions , 2013, PLoS Comput. Biol..

[33]  M. Buffelli,et al.  The timing of impulse activity shapes the process of synaptic competition at the neuromuscular junction , 2010, Neuroscience.

[34]  Henry Markram,et al.  On the computational power of circuits of spiking neurons , 2004, J. Comput. Syst. Sci..

[35]  David Kappel,et al.  A Dynamic Connectome Supports the Emergence of Stable Computational Function of Neural Circuits through Reward-Based Learning , 2017, eNeuro.

[36]  Manish Kumar.  Large-scale neuromorphic computing systems , 2016 .

[37]  Bruno A Olshausen,et al.  Sparse coding of sensory inputs , 2004, Current Opinion in Neurobiology.

[38]  Matthew S. Grubb,et al.  Adult neurogenesis and functional plasticity in neuronal circuits , 2006, Nature Reviews Neuroscience.

[39]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[40]  C. Spence,et al.  Multisensory Integration: Space, Time and Superadditivity , 2005, Current Biology.

[41]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[42]  Alan F. Murray,et al.  Synaptic rewiring for topographic mapping and receptive field development , 2010, Neural Networks.

[43]  Jon H Kaas,et al.  Topographic Maps are Fundamental to Sensory Processing , 1997, Brain Research Bulletin.

[44]  Subhrajit Roy,et al.  An Online Unsupervised Structural Plasticity Algorithm for Spiking Neural Networks , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[45]  W. Grueber,et al.  Self-avoidance and tiling: Mechanisms of dendrite and axon spacing. , 2010, Cold Spring Harbor perspectives in biology.

[46]  Rodney J. Douglas,et al.  A pulse-coded communications infrastructure for neuromorphic systems , 1999 .

[47]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[48]  P. Rakić,et al.  Synaptogenesis in monkey somatosensory cortex. , 1991, Cerebral cortex.

[49]  Jim D. Garside,et al.  On-chip order-exploiting routing table minimization for a multicast supercomputer network , 2016, 2016 IEEE 17th International Conference on High Performance Switching and Routing (HPSR).

[50]  H. Markram,et al.  The neocortical microcircuit as a tabula rasa. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[51]  Lewi Stone,et al.  Modeling the Impact of White-Plague Coral Disease in Climate Change Scenarios , 2015, PLoS Comput. Biol..

[52]  Subhrajit Roy,et al.  Liquid State Machine With Dendritically Enhanced Readout for Low-Power, Neuromorphic VLSI Implementations , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[53]  David Kappel,et al.  Network Plasticity as Bayesian Inference , 2015, PLoS Comput. Biol..

[54]  Richard Axel,et al.  Visualizing an Olfactory Sensory Map , 1996, Cell.

[55]  M. Buffelli,et al.  Activity-dependent synaptic competition at mammalian neuromuscular junctions. , 2004, News in physiological sciences : an international journal of physiology produced jointly by the International Union of Physiological Sciences and the American Physiological Society.

[56]  L. Abbott,et al.  Cortical Development and Remapping through Spike Timing-Dependent Plasticity , 2001, Neuron.

[57]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[58]  Jim D. Garside,et al.  SpiNNaker—Programming Model , 2015, IEEE Transactions on Computers.

[59]  N. Spruston,et al.  Action potential initiation and backpropagation in neurons of the mammalian CNS , 1997, Trends in Neurosciences.

[60]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[61]  B Kolb,et al.  Persistent Structural Modifications in Nucleus Accumbens and Prefrontal Cortex Neurons Produced by Previous Experience with Amphetamine , 1997, The Journal of Neuroscience.

[62]  Bernhard Hellwig,et al.  A quantitative analysis of the local connectivity between pyramidal neurons in layers 2/3 of the rat visual cortex , 2000, Biological Cybernetics.

[63]  Silvestro Micera,et al.  Multi-level imaging of brain plasticity after stroke , 2016 .

[64]  D. Chklovskii,et al.  Geometry and Structural Plasticity of Synaptic Connectivity , 2002, Neuron.

[65]  Steve B. Furber,et al.  Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[66]  Noam E. Ziv,et al.  Relative Contributions of Specific Activity Histories and Spontaneous Processes to Size Remodeling of Glutamatergic Synapses , 2016, PLoS biology.

[67]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.