SpiNNaker: impact of traffic locality, causality and burstiness on the performance of the interconnection network

The SpiNNaker system is a biologically-inspired massively parallel architecture of bespoke multi-core System-on-Chips. The aim of its design is to simulate up to a billion spiking neurons in (biological) real-time. Packets, in SpiNNaker, represent neural spikes and these travel through the two-dimensional triangular torus network that connects the over 65 thousand nodes housed in the largest size of SpiNNaker. The research question that we explore is the impact that spatial locality, temporal causality and burstiness of the traffic have on the performance of such interconnection network. Given the limited knowledge of neuron activity patterns, we propose and use synthetic traffic patterns which resemble biological neural traffic and allow tuning of spatial locality. Causality is explored by means of temporal patterns that maintain a specified overall network load while allowing at the node level autonomous causal traffic generation. Part of the traffic is generated automatically, but the remaining traffic is triggered by a spike arrival in the form of a packet or a burst of packets; as neural stimuli do. In this way, we generate non-uniform traffic patterns with an evolving concentration of activity at nodes which contain more active parts of the spiking neural network. Given the application domain, the simulation-based study focuses on the real-time behavior of the system rather than focusing on standard HPC network metrics. The results show that the interconnection network of SpiNNaker can operate without dropping packets with traffic loads that exceed more than 3.5 times those required to simulate 109 spiking neurons, despite using non-local traffic. We also find that increments in the degree of traffic causality do not affect the performance of the system, but burstiness in the traffic can hurt performance.

[1]  A. R. McIntosh,et al.  The effects of physiologically plausible connectivity structure on local and global dynamics in large scale brain models , 2009, Journal of Neuroscience Methods.

[2]  Steve B. Furber,et al.  Event-Driven Configuration of a Neural Network CMP System over a Homogeneous Interconnect Fabric , 2009, 2009 Eighth International Symposium on Parallel and Distributed Computing.

[3]  O. Sporns,et al.  Mapping the Structural Core of Human Cerebral Cortex , 2008, PLoS biology.

[4]  Luis A. Plana,et al.  A GALS Infrastructure for a Massively Parallel Multiprocessor , 2007, IEEE Design & Test of Computers.

[5]  Luis A. Plana,et al.  SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[6]  Stephen B. Furber,et al.  Efficient modelling of spiking neural networks on a scalable chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[7]  José Miguel-Alonso,et al.  INSEE: An Interconnection Network Simulation and Evaluation Environment , 2005, Euro-Par.

[8]  Krste Asanovic,et al.  Parallel neural network training on Multi-Spert , 1997, Proceedings of 3rd International Conference on Algorithms and Architectures for Parallel Processing.

[9]  Philip Heidelberger,et al.  IBM Research Report Design and Analysis of the BlueGene/L Torus Interconnection Network , 2003 .

[10]  Andrew D. Brown,et al.  On-chip and inter-chip networks for modeling large-scale neural systems , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[11]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[12]  Steve B. Furber,et al.  An On-Chip and Inter-Chip Communications Network for the SpiNNaker Massively-Parallel Neural Net Simulator , 2008, Second ACM/IEEE International Symposium on Networks-on-Chip (nocs 2008).

[13]  Steve B. Furber,et al.  Neural Systems Engineering , 2008, Computational Intelligence: A Compendium.

[14]  Jerome A. Feldman,et al.  A supercomputer for neural computation , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[15]  M. Geike,et al.  Emulation engine for spiking neurons and adaptive synaptic weights , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[16]  Steve B. Furber,et al.  Understanding the interconnection network of SpiNNaker , 2009, ICS.