Over the last couple of years, supercomputers such as the Blue Gene/Q system JUQUEEN in Julich and the K computer in Kobe have become available for neuroscience research. These massively parallel systems open the field for a new class of scientific questions as they provide the resources to represent and simulate brain-scale networks, but they also confront the developers of simulation software with a new class of problems. Initial tests with our neuronal network simulator NEST [1] on JUGENE (the predecessor of JUQUEEN) revealed that in order to exploit the memory capacities of such machines, we needed to improve the parallelization of the fundamental data structures. To address this, we developed an analytical framework [2], which serves as a guideline for a systematic and iterative restructuring of the simulation kernel. In December 2012, the 3rd generation technology was released with NEST 2.2, which enables simulations of 108 neurons and 10,000 synapses per neuron on the K computer [3].
Even though the redesign of the fundamental data structures of NEST is driven by the demand for simulations of interacting brain areas, we do not aim at solutions tailored to a specific brain-scale model or computing architecture. Our goal is to maintain a single highly scalable code base that meets the requirements of such simulations whilst still performing well on modestly dimensioned lab clusters and even laptops.
Here, we introduce the 4th generation simulation kernel and describe the development workflow that yielded the following three major improvements: the self-collapsing connection infrastructure, which takes up significantly less memory in the case of few local targets, the compacted node infrastructure, which causes only negligible constant serial memory overhead, and the reduced memory usage of synapse objects, which does not affect the precision of synaptic state variables. The improved code does not compromise on the general usability of NEST and will be merged into the common code base to be released with NEST 2.4. We show that with the 4g technology it will be possible to simulate networks of 109 neurons and 10,000 synapses per neuron on the K computer.