Distributed simulation of polychronous and plastic spiking neural networks: strong and weak scaling of a representative mini-application benchmark executed on a small-scale commodity cluster

We introduce a natively distributed mini-application benchmark representative of plastic spiking neural network simulators. It can be used to measure performances of existing computing platforms and to drive the development of future parallel/distributed computing systems dedicated to the simulation of plastic spiking networks. The mini-application is designed to generate spiking behaviors and synaptic connectivity that do not change when the number of hardware processing nodes is varied, simplifying the quantitative study of scalability on commodity and custom architectures. Here, we present the strong and weak scaling and the profiling of the computational/communication components of the DPSNN-STDP benchmark (Distributed Simulation of Polychronous Spiking Neural Network with synaptic Spike-Timing Dependent Plasticity). In this first test, we used the benchmark to exercise a small-scale cluster of commodity processors (varying the number of used physical cores from 1 to 128). The cluster was interconnected through a commodity network. Bidimensional grids of columns composed of Izhikevich neurons projected synapses locally and toward first, second and third neighboring columns. The size of the simulated network varied from 6.6 Giga synapses down to 200 K synapses. The code demonstrated to be fast and scalable: 10 wall clock seconds were required to simulate one second of activity and plasticity (per Hertz of average firing rate) of a network composed by 3.2 G synapses running on 128 hardware cores clocked @ 2.4 GHz. The mini-application has been designed to be easily interfaced with standard and custom software and hardware communication interfaces. It has been designed from its foundation to be natively distributed and parallel, and should not pose major obstacles against distribution and parallelization on several platforms.

[1]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[2]  Nicholas T. Carnevale,et al.  The NEURON Book: Epilogue , 2006 .

[3]  Raffaele Tripiccione,et al.  From APE to APE-100: From 1 to 100 gflops in lattice gauge theory simulations , 1989 .

[4]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[5]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[6]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[7]  Stephen B. Furber,et al.  Efficient modelling of spiking neural networks on a scalable chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  V. Han,et al.  Synaptic plasticity in a cerebellum-like structure depends on temporal order , 1997, Nature.

[9]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[10]  James M. Bower,et al.  GENESIS, The GEneral NEural SImulation System , 2014, Encyclopedia of Computational Neuroscience.

[11]  G. Edelman,et al.  Large-scale model of mammalian thalamocortical systems , 2008, Proceedings of the National Academy of Sciences.

[12]  Paolo Del Giudice,et al.  Efficient Event-Driven Simulation of Large Networks of Spiking Neurons and Dynamical Synapses , 2000, Neural Computation.

[13]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[14]  P. Paolucci.  N-BODY CLASSICAL SYSTEMS AND NEURAL NETWORKS ON A 3D SIMD MASSIVE PARALLEL PROCESSOR: APE100/QUADRICS , 1995 .

[15]  Pier Stanislao Paolucci,et al.  APEnet+: high bandwidth 3D torus direct network for petaflops scale commodity clusters , 2011, ArXiv.

[16]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[17]  James M. Bower,et al.  The Book of GENESIS , 1994, Springer New York.

[18]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[19]  J. A. Bewer.  The Book of Genesis , 2011 .

[20]  Nikil D. Dutt,et al.  Efficient simulation of large-scale Spiking Neural Networks using CUDA graphics processors , 2009, 2009 International Joint Conference on Neural Networks.

[21]  Rainer Leupers,et al.  EURETILE 2010-2012 summary: first three years of activity of the European Reference Tiled Experiment , 2013, ArXiv.

[22]  Dharmendra S. Modha.  Cognitive computing: Neuroscience, supercomputing, nanotechnology , 2011, DAC 2011.

[23]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.