CoreNEURON : An Optimized Compute Engine for the NEURON Simulator

The NEURON simulator has been developed over the past three decades and is widely used by neuroscientists to model the electrical activity of neuronal networks. Large network simulation projects using NEURON have supercomputer allocations that individually measure in the millions of core hours. Supercomputer centers are transitioning to next generation architectures and the work accomplished per core hour for these simulations could be improved by an order of magnitude if NEURON was able to better utilize those new hardware capabilities. In order to adapt NEURON to evolving computer architectures, the compute engine of the NEURON simulator has been extracted and has been optimized as a library called CoreNEURON. This paper presents the design, implementation, and optimizations of CoreNEURON. We describe how CoreNEURON can be used as a library with NEURON and then compare performance of different network models on multiple architectures including IBM BlueGene/Q, Intel Skylake, Intel MIC and NVIDIA GPU. We show how CoreNEURON can simulate existing NEURON network models with 4–7x less memory usage and 2–7x less execution time while maintaining binary result compatibility with NEURON.

[1]  Henry Markram,et al.  Rich cell-type-specific network topology in neocortical microcircuitry , 2017, Nature Neuroscience.

[2]  T. Sejnowski,et al.  [Letters to nature] , 1996, Nature.

[3]  Pierre J. Magistretti,et al.  Multi-timescale Modeling of Activity-Dependent Metabolic Coupling in the Neuron-Glia-Vasculature Ensemble , 2015, PLoS Comput. Biol..

[4]  J. Hostettler.  Introduction to the , 1983 .

[5]  Michael L. Hines,et al.  Comparison of neuronal spike exchange methods on a Blue Gene/P supercomputer , 2011, Front. Comput. Neurosci..

[6]  Brian R. Lee,et al.  Visual physiology of the layer 4 cortical circuit in silico , 2018, bioRxiv.

[7]  Markus Diesmann,et al.  Constructing Neuronal Network Models in Massively Parallel Environments , 2017, Front. Neuroinform..

[8]  Sten Grillner,et al.  Basal Ganglia Neuromodulation Over Multiple Temporal and Structural Scales—Simulations of Direct Pathway MSNs Investigate the Fast Onset of Dopaminergic Effects and Predict the Role of Kv4.2 , 2018, Front. Neural Circuits.

[9]  Martin A. Nowak,et al.  Life cycle synchronization is a viral drug resistance mechanism , 2018, PLoS Comput. Biol..

[10]  Bradford Nichols,et al.  Pthreads programming - a POSIX standard for better multiprocessing , 1996 .

[11]  Jürg Bähler,et al.  Proportionality: A Valid Alternative to Correlation for Relative Data , 2014, bioRxiv.

[12]  J. Bower,et al.  An active membrane model of the cerebellar Purkinje cell. I. Simulation of current clamps in slice. , 1994, Journal of neurophysiology.

[13]  L. Tsimring,et al.  Topological determinants of epileptogenesis in large-scale structural and functional models of the dentate gyrus derived from experimental data. , 2007, Journal of neurophysiology.

[14]  Michael L. Hines,et al.  NEURON — A Program for Simulation of Nerve Equations , 1993 .

[15]  Sonja Grün,et al.  Hybrid Scheme for Modeling Local Field Potentials from Point-Neuron Networks , 2015, BMC Neuroscience.

[16]  Daniel Marques,et al.  Implementation and Evaluation of a Scalable Application-Level Checkpoint-Recovery Scheme for MPI Programs , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[17]  Viktor K. Jirsa,et al.  Mathematical framework for large-scale brain network modeling in The Virtual Brain , 2015, NeuroImage.

[18]  G. Edelman,et al.  Large-scale model of mammalian thalamocortical systems , 2008, Proceedings of the National Academy of Sciences.

[19]  Eric Postma,et al.  Combining Structural Connectivity and Response Latencies to Model the Structure of the Visual System , 2008, PLoS Comput. Biol..

[20]  Alexander Peyser,et al.  Arbor — A Morphologically-Detailed Neural Network Simulation Library for Contemporary High-Performance Computing Architectures , 2019, 2019 27th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP).

[21]  Tobias C. Potjans,et al.  The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model , 2012, Cerebral cortex.

[22]  Michael Gschwind,et al.  The IBM Blue Gene/Q Compute Chip , 2012, IEEE Micro.

[23]  James G. King,et al.  Leveraging a Cluster-Booster Architecture for Brain-Scale Simulations , 2016, ISC.

[24]  L. Dagum,et al.  OpenMP: an industry standard API for shared-memory programming , 1998 .

[25]  Erik De Schutter,et al.  STEPS: Modeling and Simulating Complex Reaction-Diffusion Systems with Python , 2008, Frontiers in Neuroinformatics.

[26]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[27]  M. Hines,et al.  Efficient computation of branched nerve equations. , 1984, International journal of bio-medical computing.

[28]  A. Hindmarsh,et al.  CVODE, a stiff/nonstiff ODE solver in C , 1996 .

[29]  Mark A. Moraes,et al.  Parallel random numbers: As easy as 1, 2, 3 , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[30]  Karl J. Friston,et al.  The Dynamic Brain: From Spiking Neurons to Neural Masses and Cortical Fields , 2008, PLoS Comput. Biol..

[31]  Alexander Peyser,et al.  NestMC: A morphologically detailed neural network simulator for modern high performance computer architectures , 2016 .

[32]  C. Koch,et al.  Cell type- and activity-dependent extracellular correlates of intracellular spiking. , 2015, Journal of neurophysiology.

[33]  Lawrence Mitchell,et al.  Achieving Efficient Strong Scaling with PETSc Using Hybrid MPI/OpenMP Optimisation , 2013, ISC.

[34]  Michael L. Hines,et al.  Fully implicit parallel simulation of single neurons , 2008, Journal of Computational Neuroscience.

[35]  Philip Heidelberger,et al.  Optimization of applications with non-blocking neighborhood collectives via multisends on the Blue Gene/P supercomputer , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[36]  Shashi Prasad.  Multithreading Programming Techniques , 1997 .

[37]  S. N. Davies.  Neural Networks of the Hippocampus. By Roger D. Traub and Richard Miles. Pp. 281. Cambridge University Press, 1991. £27.50/$39.50 hardback. ISBN 0 521 36481 7 , 1992 .

[38]  Jesús Labarta,et al.  cuHinesBatch: Solving Multiple Hines systems on GPUs Human Brain Project* , 2017, ICCS.

[39]  Nicholas T. Carnevale,et al.  Expanding NEURON's Repertoire of Mechanisms with NMODL , 2000, Neural Computation.

[40]  Erik De Schutter,et al.  Accurate Reaction-Diffusion Operator Splitting on Tetrahedral Meshes for Parallel Stochastic Molecular Simulations , 2015, The Journal of chemical physics.

[41]  Bernhard Rumpe,et al.  Code Generation in Computational Neuroscience: A Review of Tools and Techniques , 2018, Front. Neuroinform..

[42]  Fiona E. N. LeBeau,et al.  Single-column thalamocortical network model exhibiting gamma oscillations, sleep spindles, and epileptogenic bursts. , 2005, Journal of neurophysiology.

[43]  James G. King,et al.  Reconstruction and Simulation of Neocortical Microcircuitry , 2015, Cell.

[44]  Michael L. Hines,et al.  Parallel network simulations with NEURON , 2006, Journal of Computational Neuroscience.

[45]  Sonja Grün,et al.  Hybrid Scheme for Modeling Local Field Potentials from Point-Neuron Networks , 2015, Cerebral cortex.

[46]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[47]  D CohenScott,et al.  CVODE, a stiff/nonstiff ODE solver in C , 1996 .

[48]  K. Shadan,et al.  Available online: , 2012 .

[49]  Achim Streit,et al.  Preparing HPC Applications for Exascale: Challenges and Recommendations , 2015, 2015 18th International Conference on Network-Based Information Systems.

[50]  Henry Markram,et al.  Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function , 2016, Front. Comput. Neurosci..