Efficient simulation of large-scale Spiking Neural Networks using CUDA graphics processors

Neural network simulators that take into account the spiking behavior of neurons are useful for studying brain mechanisms and for engineering applications. Spiking Neural Network (SNN) simulators have been traditionally simulated on large-scale clusters, super-computers, or on dedicated hardware architectures. Alternatively, Graphics Processing Units (GPUs) can provide a low-cost, programmable, and high-performance computing platform for simulation of SNNs. In this paper we demonstrate an efficient, Izhikevich neuron based large-scale SNN simulator that runs on a single GPU. The GPU-SNN model (running on an NVIDIA GTX-280 with 1GB of memory), is up to 26 times faster than a CPU version for the simulation of 100K neurons with 50 Million synaptic connections, firing at an average rate of 7Hz. For simulation of 100K neurons with 10 Million synaptic connections, the GPU-SNN model is only 1.5 times slower than real-time. Further, we present a collection of new techniques related to parallelism extraction, mapping of irregular communication, and compact network representation for effective simulation of SNNs on GPUs. The fidelity of the simulation results were validated against CPU simulations using firing rate, synaptic weight distribution, and inter-spike interval analysis. We intend to make our simulator available to the modeling community so that researchers will have easy access to large-scale SNN simulations.

[1]  Sun UltraSPARC,et al.  A closer look at GPUs , 2008, Commun. ACM.

[2]  Renaud Keriven,et al.  Spiking Neurons on GPUs , 2006, International Conference on Computational Science.

[3]  G. Edelman,et al.  Large-scale model of mammalian thalamocortical systems , 2008, Proceedings of the National Academy of Sciences.

[4]  Luis A. Plana,et al.  SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[5]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[6]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[7]  Nikil D. Dutt,et al.  Computing spike-based convolutions on GPUs , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[8]  Tim Schönauer,et al.  Simulation of Spiking Neural Networks on Different Hardware Platforms , 1997, ICANN.

[9]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[10]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[11]  L. Abbott,et al.  Cortical Development and Remapping through Spike Timing-Dependent Plasticity , 2001, Neuron.

[12]  Bertram E. Shi,et al.  Expandable Networks for Neuromorphic Chips , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Marc-Oliver Gewaltig,et al.  Efficient Parallel Simulation of Large-Scale Neuronal Networks on Clusters of Multiprocessor Computers , 2007, Euro-Par.

[14]  Dharmendra S. Modha,et al.  Anatomy of a cortical simulator , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[15]  Arnaud Delorme,et al.  Spike-based strategies for rapid processing , 2001, Neural Networks.

[16]  Wolf Singer.  Binding by synchrony , 2007, Scholarpedia.

[17]  Stephen B. Furber,et al.  Efficient modelling of spiking neural networks on a scalable chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[18]  Ernst Niebur,et al.  Efficient Simulation of Biological Neural Networks on Massively Parallel Supercomputers with Hypercube Architecture , 1993, NIPS.

[19]  Jesper Tegnér,et al.  Spike-timing-dependent plasticity: common themes and divergent vistas , 2002, Biological Cybernetics.

[20]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[21]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[22]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[23]  Gert Cauwenberghs,et al.  A Multichip Neuromorphic System for Spike-Based Visual Information Processing , 2007, Neural Computation.