Fast Simulations of Highly-Connected Spiking Cortical Models Using GPUs

Over the past decade there has been a growing interest in the development of parallel hardware systems for simulating large-scale networks of spiking neurons. Compared to other highly-parallel systems, GPU-accelerated solutions have the advantage of a relatively low cost and a great versatility, thanks also to the possibility of using the CUDA-C/C++ programming languages. NeuronGPU is a GPU library for large-scale simulations of spiking neural network models, written in the C++ and CUDA-C++ programming languages, based on a novel spike-delivery algorithm. This library includes simple LIF (leaky-integrate-and-fire) neuron models as well as several multisynapse AdEx (adaptive-exponential-integrate-and-fire) neuron models with current or conductance based synapses, different types of spike generators, tools for recording spikes, state variables and parameters, and it supports user-definable models. The numerical solution of the differential equations of the dynamics of the AdEx models is performed through a parallel implementation, written in CUDA-C++, of the fifth-order Runge-Kutta method with adaptive step-size control. In this work we evaluate the performance of this library on the simulation of a cortical microcircuit model, based on LIF neurons and current-based synapses, and on balanced networks of excitatory and inhibitory neurons, using AdEx or Izhikevich neuron models and conductance-based or current-based synapses. On these models, we will show that the proposed library achieves state-of-the-art performance in terms of simulation time per second of biological activity. In particular, using a single NVIDIA GeForce RTX 2080 Ti GPU board, the full-scale cortical-microcircuit model, which includes about 77,000 neurons and 3 · 108 connections, can be simulated at a speed very close to real time, while the simulation time of a balanced network of 1,000,000 AdEx neurons with 1,000 connections per neuron was about 70 s per second of biological activity.

[1]  Stefan Rotter,et al.  Exact digital simulation of time-invariant linear systems with applications to neuronal modeling , 1999, Biological Cybernetics.

[2]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model , 2009, Scholarpedia.

[3]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[4]  Michael L. Hines,et al.  The NEURON Book: Frontmatter , 2006 .

[5]  Roman Rybka,et al.  On the applicability of STDP-based learning mechanisms to spiking neuron network models , 2016 .

[6]  Nikil D. Dutt,et al.  CARLsim 4: An Open Source Library for Large Scale, Biologically Detailed Spiking Neural Network Simulation using Heterogeneous Clusters , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[7]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[8]  Jie Cheng,et al.  CUDA by Example: An Introduction to General-Purpose GPU Programming , 2010, Scalable Comput. Pract. Exp..

[9]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[10]  Tobias C. Potjans,et al.  The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model , 2012, Cerebral cortex.

[11]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Jason S. Rothman,et al.  Modeling Synapses , 2014, Encyclopedia of Computational Neuroscience.

[14]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[15]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[16]  William H. Press,et al.  Adaptive Stepsize Runge‐Kutta Integration , 1992 .

[17]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.

[18]  André van Schaik,et al.  An FPGA-Based Massively Parallel Neuromorphic Cortex Simulator , 2018, Front. Neurosci..

[19]  Michael L. Hines,et al.  The NEURON Book , 2006 .

[20]  Thomas Nowotny,et al.  GeNN: a code generation framework for accelerated brain simulations , 2016, Scientific Reports.

[21]  Eduardo Ros,et al.  Event and Time Driven Hybrid Simulation of Spiking Neural Networks , 2011, IWANN.

[22]  Nikil D. Dutt,et al.  A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors , 2009, Neural Networks.

[23]  Julien Vitay,et al.  ANNarchy: a code generation approach to neural simulations on parallel hardware , 2015, Front. Neuroinform..

[24]  Steve B. Furber,et al.  Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model , 2018, Front. Neurosci..

[25]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[26]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[27]  Stanley C. Eisenstat,et al.  Yale sparse matrix package I: The symmetric codes , 1982 .

[28]  Thomas Nowotny,et al.  GPUs Outperform Current HPC and Neuromorphic Solutions in Terms of Speed and Energy When Simulating a Highly-Connected Cortical Model , 2018, Front. Neurosci..

[29]  Romain Brette,et al.  Simulating spiking neural networks on GPU , 2012, Network.

[30]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[31]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[32]  Romain Brette,et al.  Neuroinformatics Original Research Article Brian: a Simulator for Spiking Neural Networks in Python , 2022 .