A hierachical configuration system for a massively parallel neural hardware platform

Simulation of large networks of neurons is a powerful and increasingly prominent methodology for investigate brain functions and structures. Dedicated parallel hardware is a natural candidate for simulating the dynamic activity of many non-linear units communicating asynchronously. It is only scientifically useful, however, if the simulation tools can be configured and run easily and quickly. We present a method to map network models to computational nodes on the SpiNNaker system, a programmable parallel neurally-inspired hardware architecture, by exploiting the hierarchies built in the model. This PArtitioning and Configuration MANager (PACMAN) system supports arbitrary network topologies and arbitrary membrane potential and synapse dynamics, and (most importantly) decouples the model from the device, allowing a variety of languages (PyNN, Nengo, etc.) to drive the simulation hardware. Model representation operates on a Population/Projection level rather than a single-neuron and connection level, exploiting hierarchical properties to lower the complexity of allocating resources and mapping the model onto the system. PACMAN can be thus be used to generate structures coming from different models and front-ends, either with a host-based process, or by parallelising it on the SpiNNaker machine itself to speed up the generation process greatly. We describe the approach with a first implementation of the framework used to configure the current generation of SpiNNaker machines and present results from a set of key benchmarks. The system allows researchers to exploit dedicated simulation hardware which may otherwise be difficult to program. In effect, PACMAN provides automated hardware acceleration for some commonly used network simulators while also pointing towards the advantages of hierarchical configuration for large, domain-specific hardware systems.

[1]  Kwabena Boahen,et al.  Neuronal ion-channel dynamics in silicon , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[2]  Johannes Schemmel,et al.  A comprehensive workflow for general-purpose neural modeling with highly configurable neuromorphic hardware systems , 2010, Biological Cybernetics.

[3]  Vivek K. Pallipuram,et al.  Acceleration of spiking neural networks in emerging multi-core and GPU architectures , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW).

[4]  Yaroslav O. Halchenko,et al.  Neuroscience Runs on GNU/Linux , 2011, Front. Neuroinform..

[5]  Andrew D. Brown,et al.  On-chip and inter-chip networks for modeling large-scale neural systems , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[6]  Steve B. Furber,et al.  Event-Driven Simulation of Arbitrary Spiking Neural Networks on SpiNNaker , 2011, ICONIP.

[7]  Francesco Galluppi,et al.  A forecast-based STDP rule suitable for neuromorphic implementation , 2012, Neural Networks.

[8]  Tim P Vogels,et al.  Signal Propagation and Logic Gating in Networks of Integrate-and-Fire Neurons , 2005, The Journal of Neuroscience.

[9]  Ammar Belatreche,et al.  Challenges for large-scale implementations of spiking neural networks on FPGAs , 2007, Neurocomputing.

[10]  Anders Lansner,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[11]  Henry Markram,et al.  On the computational power of circuits of spiking neurons , 2004, J. Comput. Syst. Sci..

[12]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[13]  G. Edelman,et al.  Complexity and coherency: integrating information in the brain , 1998, Trends in Cognitive Sciences.

[14]  Bertram E. Shi,et al.  Expandable Networks for Neuromorphic Chips , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[15]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[16]  J. Bower,et al.  The Book of GENESIS , 1998, Springer New York.

[17]  Jan van Leeuwen,et al.  Interval Routing , 1987, Computer/law journal.

[18]  Steve B. Furber,et al.  A General-Purpose Model Translation System for a Universal Neural Chip , 2010, ICONIP.

[19]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[20]  Jim D. Garside,et al.  Scalable communications for a million-core neural processing architecture , 2012, J. Parallel Distributed Comput..

[21]  Luis A. Plana,et al.  A GALS Infrastructure for a Massively Parallel Multiprocessor , 2007, IEEE Design & Test of Computers.

[22]  Andrew D. Brown,et al.  Biologically-Inspired Massively-Parallel Architectures - Computing Beyond a Million Processors , 2009, ACSD.

[23]  Marc-Oliver Gewaltig,et al.  Efficient Parallel Simulation of Large-Scale Neuronal Networks on Clusters of Multiprocessor Computers , 2007, Euro-Par.

[24]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[25]  Moritz Helias,et al.  PyNEST: A Convenient Interface to the NEST Simulator , 2008, Front. Neuroinform..

[26]  Terrence C. Stewart,et al.  Python Scripting in the Nengo Simulator , 2008, Front. Neuroinform..

[27]  Dan F. M. Goodman.  Code Generation: A Strategy for Neural Network Simulators , 2010, Neuroinformatics.

[28]  Murray Shanahan,et al.  NeMo: A Platform for Neural Modelling of Spiking Neurons Using GPUs , 2009, 2009 20th IEEE International Conference on Application-specific Systems, Architectures and Processors.

[29]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[30]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[31]  Gert Cauwenberghs,et al.  A Multichip Neuromorphic System for Spike-Based Visual Information Processing , 2007, Neural Computation.

[32]  Romain Brette,et al.  Neuroinformatics Original Research Article Brian: a Simulator for Spiking Neural Networks in Python , 2022 .

[33]  Bertram E. Shi,et al.  An ON-OFF orientation selective address event representation image transceiver chip , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[34]  Steve B. Furber,et al.  The Leaky Integrate-and-Fire neuron: A platform for synaptic model exploration on the SpiNNaker chip , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[35]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[36]  James M. Bower,et al.  The book of GENESIS - exploring realistic neural models with the GEneral NEural SImulation System (2. ed.) , 1994 .

[37]  Andrew S. Cassidy,et al.  Design of a one million neuron single FPGA neuromorphic system for real-time multimodal scene analysis , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[38]  Steve B. Furber,et al.  Real time on-chip implementation of dynamical systems with spiking neurons , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[39]  Ran Ginosar,et al.  Scalable network-on-chip architecture for configurable neural networks , 2011, Microprocess. Microsystems.

[40]  John Wawrzynek,et al.  Silicon Auditory Processors as Computer Peripherals , 1992, NIPS.

[41]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[42]  A. Aertsen,et al.  Gating of signal propagation in spiking neural networks by balanced and correlated excitation and inhibition , 2010 .

[43]  Nigel H. Goddard,et al.  Towards NeuroML: model description methods for collaborative modelling in neuroscience. , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[44]  Steve Furber,et al.  Power-efficient simulation of detailed cortical microcircuits on SpiNNaker , 2012, Journal of Neuroscience Methods.

[45]  Steve Furber,et al.  High-performance computing for systems of spiking neurons , 2006 .

[46]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[47]  Dharmendra S. Modha,et al.  The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[48]  Nikil D. Dutt,et al.  A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors , 2009, Neural Networks.