SpiNNaker: Enhanced multicast routing

Abstract The human brain is a complex biological neural network characterised by high degrees of connectivity among neurons. Any system designed to simulate large-scale spiking neuronal networks needs to support such connectivity and the associated communication traffic in the form of spike events. This paper investigates how best to generate multicast routes for SpiNNaker, a purpose-built, low-power, massively-parallel architecture. The discussed algorithms are an essential ingredient for the efficient operation of SpiNNaker since generating multicast routes is known to be an NP-complete problem. In fact, multicast communications have been extensively studied in the literature, but we found no existing algorithm adaptable to SpiNNaker. The proposed algorithms exploit the regularity of the two-dimensional triangular torus topology and the availability of selective multicast at hardware level. A comprehensive study of the parameters of the algorithms and their effectiveness is carried out in this paper considering different destination distributions ranging from worst-case to a real neural application. The results show that two novel proposed algorithms can reduce significantly the pressure exerted onto the interconnection infrastructure while remaining effective to be used in a production environment.

[1]  Steve B. Furber,et al.  Interfacing Real-Time Spiking I/O with the SpiNNaker Neuromimetic Architecture , 2010, Aust. J. Intell. Inf. Process. Syst..

[2]  William J. Dally,et al.  Principles and Practices of Interconnection Networks , 2004 .

[3]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[4]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[5]  J. Orbach.  Principles of Neurodynamics. Perceptrons and the Theory of Brain Mechanisms. , 1962 .

[6]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[7]  Steve B. Furber,et al.  On generating multicast routes for SpiNNaker , 2014, Conf. Computing Frontiers.

[8]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[9]  Tomohiro Inoue,et al.  The Tofu Interconnect , 2012, IEEE Micro.

[10]  Rajeev Rastogi,et al.  Topology discovery in heterogeneous IP networks: the NetInventory system , 2004, IEEE/ACM Transactions on Networking.

[11]  José E. Moreira,et al.  IBM Blue Gene Supercomputer , 2011, Encyclopedia of Parallel Computing.

[12]  Sadaf R. Alam,et al.  Cray XT4: an early evaluation for petascale scientific simulation , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[13]  Krste Asanovic,et al.  Parallel neural network training on Multi-Spert , 1997, Proceedings of 3rd International Conference on Algorithms and Architectures for Parallel Processing.

[14]  Stephen B. Furber,et al.  Efficient modelling of spiking neural networks on a scalable chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[15]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[16]  Javier Navaridas,et al.  Analytical Assessment of the Suitability of Multicast Communications for the SpiNNaker Neuromimetic System , 2012, 2012 IEEE 14th International Conference on High Performance Computing and Communication & 2012 IEEE 9th International Conference on Embedded Software and Systems.

[17]  Andrew D. Brown,et al.  On-chip and inter-chip networks for modeling large-scale neural systems , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[18]  Steve B. Furber,et al.  Understanding the interconnection network of SpiNNaker , 2009, ICS.

[19]  Collin McCurdy,et al.  Early evaluation of IBM BlueGene/P , 2008, HiPC 2008.

[20]  S. Herculano‐Houzel.  The Human Brain in Numbers: A Linearly Scaled-up Primate Brain , 2009, Front. Hum. Neurosci..

[21]  S. Herculano‐Houzel,et al.  Cellular scaling rules for rodent brains , 2006, Proceedings of the National Academy of Sciences.

[22]  Hyungjun Kim,et al.  Recursive partitioning multicast: A bandwidth-efficient routing for Networks-on-Chip , 2009, 2009 3rd ACM/IEEE International Symposium on Networks-on-Chip.

[23]  R. Menzel,et al.  Cognitive architecture of a mini-brain: the honeybee , 2001, Trends in Cognitive Sciences.

[24]  Terry Elliott,et al.  Developmental robotics: manifesto and application , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[25]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[26]  Jim Kurose,et al.  Topology discovery service for router-assisted multicast transport , 2002, 2002 IEEE Open Architectures and Network Programming Proceedings. OPENARCH 2002 (Cat. No.02EX571).

[27]  O. Sporns,et al.  Mapping the Structural Core of Human Cerebral Cortex , 2008, PLoS biology.

[28]  Natalie D. Enright Jerger,et al.  Virtual Circuit Tree Multicasting: A Case for On-Chip Hardware Multicast Support , 2008, 2008 International Symposium on Computer Architecture.

[29]  Dhabaleswar K. Panda,et al.  Multidestination Message Passing in Wormhole k-ary n-cube Networks with Base Routing Conformed Paths , 1999, IEEE Trans. Parallel Distributed Syst..

[30]  Kevan A. C. Martin,et al.  Topology and dynamics of the canonical circuit of cat V1 , 2009, Neural Networks.

[31]  Henry Markram,et al.  The neocortical column. , 2012, Frontiers in neuroanatomy.

[32]  Christof Koch,et al.  Introduction to methods in neuronal modeling , 1989 .

[33]  Johannes Schemmel,et al.  Realizing biological spiking network models in a configurable wafer-scale hardware system , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[34]  Yu-Chee Tseng,et al.  A Trip-Based Multicasting Model in Wormhole-Routed Networks with Virtual Channels , 1996, IEEE Trans. Parallel Distributed Syst..

[35]  Steve B. Furber,et al.  Managing Burstiness and Scalability in Event-Driven Models on the SpiNNaker Neuromimetic System , 2011, International Journal of Parallel Programming.

[36]  W. Daniel Hillis,et al.  The CM-5 Connection Machine: a scalable supercomputer , 1993, CACM.

[37]  David F. Heidel,et al.  An Overview of the BlueGene/L Supercomputer , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[38]  Xiaola Lin,et al.  Deadlock-Free Multicast Wormhole Routing in 2-D Mesh Multicomputers , 1994, IEEE Trans. Parallel Distributed Syst..

[39]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[40]  José Duato,et al.  Efficient and Scalable Hardware-Based Multicast in Fat-Tree Networks , 2009, IEEE Transactions on Parallel and Distributed Systems.

[41]  Xiaola Lin,et al.  Multicast Communication in Multicomputer Networks , 1993, ICPP.

[42]  Steve B. Furber,et al.  Population-based routing in the SpiNNaker neuromorphic architecture , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[43]  Steve B. Furber,et al.  A Multicast Routing Scheme for a Universal Spiking Neural Network Architecture , 2010, Comput. J..

[44]  A. Thomson,et al.  Functional Maps of Neocortical Local Circuitry , 2007, Front. Neurosci..

[45]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[46]  Jerome A. Feldman,et al.  A supercomputer for neural computation , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[47]  Wei-Tek Tsai,et al.  Multicasting in Generalized Multistage Interconnection Networks , 1994, J. Parallel Distributed Comput..