A new GPU library for fast simulation of large-scale networks of spiking neurons

Over the past decade there has been a growing interest in the development of parallel hardware systems for simulating large-scale networks of spiking neurons. Compared to other highly-parallel systems, GPU-accelerated solutions have the advantage of a relatively low cost and a great versatility, thanks also to the possibility of using the CUDA-C/C++ programming languages. NeuronGPU is a GPU library for large-scale simulations of spiking neural network models, written in the C++ and CUDA-C++ programming languages, based on a novel spike-delivery algorithm. This library includes simple LIF (leaky-integrate-and-fire) neuron models as well as several multisynapse AdEx (adaptive-exponential-integrate-and-fire) neuron models with current or conductance based synapses, user definable models and different devices. The numerical solution of the differential equations of the dynamics of the AdEx models is performed through a parallel implementation, written in CUDA-C++, of the fifth-order Runge-Kutta method with adaptive step-size control. In this work we evaluate the performance of this library on the simulation of a well-known cortical microcircuit model, based on LIF neurons and current-based synapses, and on a balanced networks of excitatory and inhibitory neurons, using AdEx neurons and conductance-based synapses. On these models, we will show that the proposed library achieves state-of-the-art performance in terms of simulation time per second of biological activity. In particular, using a single NVIDIA GeForce RTX 2080 Ti GPU board, the full-scale cortical-microcircuit model, which includes about 77,000 neurons and $3 \cdot 10^8$ connections, can be simulated at a speed very close to real time, while the simulation time of a balanced network of 1,000,000 AdEx neurons with 1,000 connections per neuron was about 70 s per second of biological activity.

[1]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[2]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[3]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[4]  William H. Press,et al.  Adaptive Stepsize Runge‐Kutta Integration , 1992 .

[5]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[6]  Stefan Rotter,et al.  Exact digital simulation of time-invariant linear systems with applications to neuronal modeling , 1999, Biological Cybernetics.

[7]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[8]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[9]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[10]  Jie Cheng,et al.  CUDA by Example: An Introduction to General-Purpose GPU Programming , 2010, Scalable Comput. Pract. Exp..

[11]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[12]  Eduardo Ros,et al.  Event and Time Driven Hybrid Simulation of Spiking Neural Networks , 2011, IWANN.

[13]  Romain Brette,et al.  Simulating spiking neural networks on GPU , 2012, Network.

[14]  Jason S. Rothman,et al.  Modeling Synapses , 2014, Encyclopedia of Computational Neuroscience.

[15]  Tobias C. Potjans,et al.  The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model , 2012, Cerebral cortex.

[16]  Julien Vitay,et al.  ANNarchy: a code generation approach to neural simulations on parallel hardware , 2015, Front. Neuroinform..

[17]  Steve B. Furber,et al.  Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model , 2018, Front. Neurosci..

[18]  Thomas Nowotny,et al.  GPUs Outperform Current HPC and Neuromorphic Solutions in Terms of Speed and Energy When Simulating a Highly-Connected Cortical Model , 2018, Front. Neurosci..

[19]  André van Schaik,et al.  An FPGA-Based Massively Parallel Neuromorphic Cortex Simulator , 2018, Front. Neurosci..

[20]  Liz Campbell,et al.  Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model , 2018 .

[21]  Nikil D. Dutt,et al.  CARLsim 4: An Open Source Library for Large Scale, Biologically Detailed Spiking Neural Network Simulation using Heterogeneous Clusters , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).