Extremely Scalable Spiking Neuronal Network Simulation Code: From Laptops to Exascale Computers

State-of-the-art software tools for neuronal network simulations scale to the largest computing systems available today and enable investigations of large-scale networks of up to 10 % of the human cortex at a resolution of individual neurons and synapses. Due to an upper limit on the number of incoming connections of a single neuron, network connectivity becomes extremely sparse at this scale. To manage computational costs, simulation software ultimately targeting the brain scale needs to fully exploit this sparsity. Here we present a two-tier connection infrastructure and a framework for directed communication among compute nodes accounting for the sparsity of brain-scale networks. We demonstrate the feasibility of this approach by implementing the technology in the NEST simulation code and we investigate its performance in different scaling scenarios of typical network simulations. Our results show that the new data structures and communication scheme prepare the simulation kernel for post-petascale high-performance computing facilities without sacrificing performance in smaller systems.

[1]  Jon Louis Bentley,et al.  Writing efficient programs , 1982 .

[2]  T. M. Mayhew,et al.  Anatomy of the Cortex: Statistics and Geometry. , 1991 .

[3]  John Shalf,et al.  The International Exascale Software Project roadmap , 2011, Int. J. High Perform. Comput. Appl..

[4]  Moritz Helias,et al.  Integration of Continuous-Time Dynamics in a Spiking Neural Network Simulator , 2016, Front. Neuroinform..

[5]  James Kozloski,et al.  An Ultrascalable Solution to Large-scale Neural Tissue Simulation , 2011, Front. Neuroinform..

[6]  James G. King,et al.  Leveraging a Cluster-Booster Architecture for Brain-Scale Simulations , 2016, ISC.

[7]  Bernhard Rumpe,et al.  NESTML: a modeling language for spiking neurons , 2016, Modellierung.

[8]  Abigail Morrison,et al.  CyNEST: a maintainable Cython-based interface for the NEST simulator , 2014, Front. Neuroinform..

[9]  Graph Topology.  MPI at Exascale , 2010 .

[10]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[11]  Gerhard Wellein,et al.  Introduction to High Performance Computing for Scientists and Engineers , 2010, Chapman and Hall / CRC computational science series.

[12]  Claes Johnson.  Numerical solution of partial differential equations by the finite element method , 1988 .

[13]  Franck Cappello,et al.  Toward Exascale Resilience: 2014 update , 2014, Supercomput. Front. Innov..

[14]  Marc-Oliver Gewaltig,et al.  Towards Reproducible Descriptions of Neuronal Network Models , 2009, PLoS Comput. Biol..

[15]  Marc-Oliver Gewaltig,et al.  SYNOD: An Environment for Neural Systems Simulations Language Interface and Tutorial , 2007 .

[16]  Trevor Bekolay,et al.  Nengo: a Python tool for building large-scale functional brain models , 2014, Front. Neuroinform..

[17]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[18]  Viktor K. Jirsa,et al.  The Virtual Brain: a simulator of primate brain network dynamics , 2013, Front. Neuroinform..

[19]  Tomoki Fukai,et al.  Spiking network simulation code for petascale computers , 2014, Front. Neuroinform..

[20]  Moritz Helias,et al.  Scalability of Asynchronous Networks Is Limited by One-to-One Mapping between Effective Connectivity and Correlations , 2014, PLoS Comput. Biol..

[21]  Marc-Oliver Gewaltig,et al.  NEST: An Environment for Neural Systems Simulations , 2003 .

[22]  Michael L. Hines,et al.  Comparison of neuronal spike exchange methods on a Blue Gene/P supercomputer , 2011, Front. Comput. Neurosci..

[23]  J. Fischer.  Principles of Neural Science (3rd ed.) , 1993 .

[24]  Moritz Helias,et al.  Neuroinformatics Original Research Article Pynest: a Convenient Interface to the Nest Simulator , 2022 .

[25]  Markus Diesmann,et al.  Constructing Neuronal Network Models in Massively Parallel Environments , 2017, Front. Neuroinform..

[26]  Nikola T. Markov,et al.  A Weighted and Directed Interareal Connectivity Matrix for Macaque Cerebral Cortex , 2012, Cerebral cortex.

[27]  Bernhard Hellwig,et al.  A quantitative analysis of the local connectivity between pyramidal neurons in layers 2/3 of the rat visual cortex , 2000, Biological Cybernetics.

[28]  Markus Diesmann,et al.  Integrating Brain Structure and Dynamics on Supercomputers , 2013, BrainComp.

[29]  Wolfram Schenck,et al.  Performance Model for Large – Scale Neural Simulations with NEST , 2014 .

[30]  Jutta Docter,et al.  JUQUEEN: IBM Blue Gene/Q® Supercomputer System at the Jülich Supercomputing Centre , 2015 .

[31]  Romain Brette,et al.  The Brian Simulator , 2009, Front. Neurosci..

[32]  Markus Diesmann,et al.  Meeting the Memory Challenges of Brain-Scale Network Simulation , 2012, Front. Neuroinform..

[33]  Erik De Schutter,et al.  STEPS: Modeling and Simulating Complex Reaction-Diffusion Systems with Python , 2008, Frontiers in Neuroinformatics.

[34]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[35]  Tomoki Fukai,et al.  Supercomputers Ready for Use as Discovery Machines for Neuroscience , 2012, Front. Neuroinform..

[36]  Wolfram Schenck,et al.  The NEST Dry-Run Mode: Efficient Dynamic Analysis of Neuronal Network Simulation Code , 2017, Front. Neuroinform..

[37]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[38]  James M. Bower,et al.  Genesis , 2007, Scholarpedia.

[39]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[40]  S. Bressler,et al.  Large-scale brain networks in cognition: emerging methods and principles , 2010, Trends in Cognitive Sciences.

[41]  Michael L. Hines,et al.  The NEURON Book , 2006 .

[42]  Moritz Helias,et al.  A unified framework for spiking and gap-junction interactions in distributed neuronal network simulations , 2015, Front. Neuroinform..

[43]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[44]  Abigail Morrison,et al.  Automatic Generation of Connectivity for Large-Scale Neuronal Network Models through Structural Plasticity , 2016, Front. Neuroanat..

[45]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[46]  Michael L. Hines,et al.  Simulation Neurotechnologies for Advancing Brain Research: Parallelizing Large Networks in NEURON , 2016, Neural Computation.

[47]  Anders Lansner,et al.  Virtues, Pitfalls, and Methodology of Neuronal Network Modeling and Simulations on Supercomputers , 2012 .

[48]  Alex S. Ferecskó,et al.  The fractions of short- and long-range connections in the visual cortex , 2009, Proceedings of the National Academy of Sciences.

[49]  Fumiyoshi Shoji,et al.  Overview of the K computer System , 2012 .

[50]  Markus Diesmann,et al.  Frontiers in Computational Neuroscience Enabling Functional Neural Circuit Simulations with Distributed Computing of Neuromodulated Plasticity , 2022 .

[51]  Tobias C. Potjans,et al.  The Cell-Type Specific Cortical Microcircuit: Relating Structure and Activity in a Full-Scale Spiking Network Model , 2012, Cerebral cortex.