Supercomputers Ready for Use as Discovery Machines for Neuroscience

NEST is a widely used tool to simulate biological spiking neural networks. Here we explain the improvements, guided by a mathematical model of memory consumption, that enable us to exploit for the first time the computational power of the K supercomputer for neuroscience. Multi-threaded components for wiring and simulation combine 8 cores per MPI process to achieve excellent scaling. K is capable of simulating networks corresponding to a brain area with 108 neurons and 1012 synapses in the worst case scenario of random connectivity; for larger networks of the brain its hierarchical organization can be exploited to constrain the number of communicating computer nodes. We discuss the limits of the software technology, comparing maximum filling scaling plots for K and the JUGENE BG/P system. The usability of these machines for network simulations has become comparable to running simulations on a single PC. Turn-around times in the range of minutes even for the largest systems enable a quasi interactive working style and render simulations on this scale a practical tool for computational neuroscience.

[1]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[2]  Markus Diesmann,et al.  Limits to the Development of Feed-Forward Structures in Large Recurrent Neuronal Networks , 2011, Front. Comput. Neurosci..

[3]  Alex S. Ferecskó,et al.  The fractions of short- and long-range connections in the visual cortex , 2009, Proceedings of the National Academy of Sciences.

[4]  Michael L. Hines,et al.  Comparison of neuronal spike exchange methods on a Blue Gene/P supercomputer , 2011, Front. Comput. Neurosci..

[5]  Markus Diesmann,et al.  Frontiers in Computational Neuroscience Enabling Functional Neural Circuit Simulations with Distributed Computing of Neuromodulated Plasticity , 2022 .

[6]  Markus Diesmann,et al.  Spike-Timing-Dependent Plasticity in Balanced Random Networks , 2007, Neural Computation.

[7]  Anders Lansner,et al.  Bistable, Irregular Firing and Population Oscillations in a Modular Attractor Memory Network , 2010, PLoS Comput. Biol..

[8]  Toshiyuki Shimizu,et al.  Tofu: A 6D Mesh/Torus Interconnect for Exascale Computers , 2009, Computer.

[9]  Klaus Schuch,et al.  PCSIM: A Parallel Simulation Environment for Neural Circuits Fully Integrated with Python , 2008, Frontiers Neuroinformatics.

[10]  Moritz Helias,et al.  A General and Efficient Method for Incorporating Precise Spike Times in Globally Time-Driven Simulations , 2010, Front. Neuroinform..

[11]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[12]  Örjan Ekeberg,et al.  Brain-scale simulation of the neocortex on the IBM Blue Gene/L supercomputer , 2008, IBM J. Res. Dev..

[13]  D. Amit,et al.  Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. , 1997, Cerebral cortex.

[14]  Michael L. Hines,et al.  Neuron splitting in compute-bound parallel network simulations enables runtime scaling with twice as many processors , 2008, Journal of Computational Neuroscience.

[15]  Örjan Ekeberg,et al.  Massively parallel simulation of brain-scale neuronal network models , 2005 .

[16]  Markus Diesmann,et al.  Maintaining Causality in Discrete Time Neuronal Network Simulations , 2007 .

[17]  Markus Diesmann,et al.  Exact Subthreshold Integration with Continuous Spike Times in Discrete-Time Neural Network Simulations , 2007, Neural Computation.

[18]  Robert Clewley,et al.  NineML: the network interchange for neuroscience modeling language , 2011, BMC Neuroscience.

[19]  Markus Diesmann,et al.  The cell-type specific connectivity of the local cortical network explains prominent features of neuronal activity , 2011, 1106.5678.

[20]  D. V. van Essen,et al.  Corticocortical and thalamocortical information flow in the primate visual system. , 2005, Progress in brain research.

[21]  Stefan Rotter,et al.  Exact digital simulation of time-invariant linear systems with applications to neuronal modeling , 1999, Biological Cybernetics.

[22]  Nicolas Brunel,et al.  Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons , 2000, Journal of Computational Neuroscience.

[23]  Daisuke Miyamoto,et al.  Neural Circuit Simulation of Hodgkin-Huxley Type Neurons Toward Peta Scale Computers , 2012, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.

[24]  V. Mountcastle.  The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[25]  Peter S. Pacheco.  Parallel programming with MPI , 1996 .

[26]  Olaf Sporns,et al.  The Human Connectome: A Structural Description of the Human Brain , 2005, PLoS Comput. Biol..

[27]  Örjan Ekeberg,et al.  Large Neural Network Simulations on Multiple Hardware Platforms , 1997, Journal of Computational Neuroscience.

[28]  Bjarne Stroustrup,et al.  C++ Programming Language , 1986, IEEE Softw..

[29]  Wulfram Gerstner,et al.  Phenomenological models of synaptic plasticity based on spike timing , 2008, Biological Cybernetics.

[30]  Michael L. Hines,et al.  Fully implicit parallel simulation of single neurons , 2008, Journal of Computational Neuroscience.

[31]  Felix Wolf,et al.  Scalable massively parallel I/O to task-local files , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[32]  Anders Lansner,et al.  Virtues, Pitfalls, and Methodology of Neuronal Network Modeling and Simulations on Supercomputers , 2012 .

[33]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[34]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[35]  Olaf Sporns,et al.  What Is the Human Connectome , 2009 .

[36]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[37]  M. Hines,et al.  Efficient computation of branched nerve equations. , 1984, International journal of bio-medical computing.

[38]  D. B. Leitch,et al.  Neuron densities vary across and within cortical areas in primates , 2010, Proceedings of the National Academy of Sciences.

[39]  Michael L. Hines,et al.  NeuroML: A Language for Describing Data Driven Models of Neurons and Networks with a High Degree of Biological Detail , 2010, PLoS Comput. Biol..

[40]  Philip Heidelberger,et al.  Optimization of applications with non-blocking neighborhood collectives via multisends on the Blue Gene/P supercomputer , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing (IPDPS).

[41]  Marc-Oliver Gewaltig,et al.  Efficient Parallel Simulation of Large-Scale Neuronal Networks on Clusters of Multiprocessor Computers , 2007, Euro-Par.

[42]  Marc-Oliver Gewaltig,et al.  Multithreaded and Distributed Simulation of Large Biological Neuronal Networks , 2007, PVM/MPI.

[43]  Dharmendra S. Modha,et al.  Anatomy of a cortical simulator , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[44]  Moritz Helias,et al.  Neuroinformatics Original Research Article Pynest: a Convenient Interface to the Nest Simulator , 2022 .

[45]  Mitsuhisa Sato,et al.  Advanced Institute for Computational Science (AICS): Japanese national high-performance computing research institute and its 10-petaflops supercomputer “K” , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).