Gaussian and Exponential Lateral Connectivity on Distributed Spiking Neural Network Simulation

We measured the impact of long-range exponentially decaying intra-areal lateral connectivity on the scaling and memory occupation of a distributed spiking neural network simulator compared to that of short-range Gaussian decays. Indeed, while previous studies adopted short-range connectivity, recent experimental neurosciences studies are pointing out the role of longer-range intra-areal connectivity with implications on neural simulation platforms. Two- dimensional grids of cortical columns composed by up to 11 M point-like spiking neurons with spike frequency adaption were connected by up to 30 G synapses using short- and long-range connectivity models. The MPI processes composing the distributed simulator were run on up to 1024 hardware cores, hosted on a 64 nodes server platform. The hardware platform was a cluster of IBM NX360 M5 16-core compute nodes, each one containing two Intel Xeon Haswell 8-core E5-2630 v3 processors, with a clock of 2.40G Hz, interconnected through an InfiniBand network, equipped with 4 QDR switches.

[1]  John Wawrzynek,et al.  Silicon Auditory Processors as Computer Peripherals , 1992, NIPS.

[2]  Alexander Peyser,et al.  Nest 2.12.0 , 2017 .

[3]  G. Edelman,et al.  Large-scale model of mammalian thalamocortical systems , 2008, Proceedings of the National Academy of Sciences.

[4]  Ad Aertsen,et al.  Physiology and Impact of Horizontal Connections in Rat Neocortex. , 2015, Cerebral cortex.

[5]  Tomoki Fukai,et al.  Spiking network simulation code for petascale computers , 2014, Front. Neuroinform..

[6]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[7]  Rainer Leupers,et al.  Dynamic many-process applications on many-tile embedded systems and HPC clusters: The EURETILE programming environment and execution platforms , 2016, J. Syst. Archit..

[8]  Y. Zhang,et al.  The ExaNeSt Project: Interconnects, Storage, and Packaging for Exascale Systems , 2016, 2016 Euromicro Conference on Digital System Design (DSD).

[9]  Nikil D. Dutt,et al.  A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors , 2009, Neural Networks.

[10]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[11]  Tobias C. Potjans,et al.  The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model , 2012, Cerebral cortex.

[12]  Maurizio Mattia,et al.  Diverse population-bursting modes of adapting spiking neurons. , 2007, Physical review letters.

[13]  Matthew A. Wilson,et al.  GENESIS: A System for Simulating Neural Networks , 1988, NIPS.

[14]  A. Litwin-Kumar,et al.  Slow dynamics and high variability in balanced cortical networks with clustered connections , 2012, Nature Neuroscience.

[15]  Maria V. Sanchez-Vives,et al.  Slow and fast rhythms generated in the cerebral cortex of the anesthetized mouse. , 2011, Journal of neurophysiology.

[16]  Robert A. Legenstein,et al.  Neuromorphic hardware in the loop: Training a deep spiking network on the BrainScaleS wafer-scale system , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[17]  A. Schüz,et al.  Quantitative aspects of corticocortical connections: a tracer study in the mouse. , 2006, Cerebral cortex.

[18]  A. Aertsen,et al.  Beyond the Cortical Column: Abundance and Physiology of Horizontal Connections Imply a Strong Role for Inputs from the Surround , 2011, Front. Neurosci..

[19]  Alex S. Ferecskó,et al.  The fractions of short- and long-range connections in the visual cortex , 2009, Proceedings of the National Academy of Sciences.

[20]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[21]  Pier Stanislao Paolucci,et al.  Power, Energy and Speed of Embedded and Server Multi-Cores applied to Distributed Simulation of Spiking Neural Networks: ARM in NVIDIA Tegra vs Intel Xeon quad-cores , 2015, ArXiv.

[22]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[23]  Peter Pagel,et al.  Cognitive Computing , 2018, Informatik-Spektrum.

[24]  Henry Markram,et al.  Rich cell-type-specific network topology in neocortical microcircuitry , 2017, Nature Neuroscience.

[25]  Paolo Del Giudice,et al.  Efficient Event-Driven Simulation of Large Networks of Spiking Neurons and Dynamical Synapses , 2000, Neural Computation.

[26]  A. Konnerth,et al.  Making Waves: Initiation and Propagation of Corticothalamic Ca2+ Waves In Vivo , 2013, Neuron.

[27]  Maria V. Sanchez-Vives,et al.  Slow Waves in Cortical Slices: How Spontaneous Activity is Shaped by Laminar Structure , 2019, Cerebral cortex.

[28]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[29]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.