Managing Burstiness and Scalability in Event-Driven Models on the SpiNNaker Neuromimetic System

Neural networks present a fundamentally different model of computation from the conventional sequential digital model, for which conventional hardware is typically poorly matched. However, a combination of model and scalability limitations has meant that neither dedicated neural chips nor FPGA’s have offered an entirely satisfactory solution. SpiNNaker introduces a different approach, the “neuromimetic” architecture, that maintains the neural optimisation of dedicated chips while offering FPGA-like universal configurability. This parallel multiprocessor employs an asynchronous event-driven model that uses interrupt-generating dedicated hardware on the chip to support real-time neural simulation. Nonetheless, event handling, particularly packet servicing, requires careful and innovative design in order to avoid local processor congestion and possible deadlock. We explore the impact that spatial locality, temporal causality and burstiness of traffic have on network performance, using tunable, biologically similar synthetic traffic patterns. Having established the viability of the system for real-time operation, we use two exemplar neural models to illustrate how to implement efficient event-handling service routines that mitigate the problem of burstiness in the traffic. Extending work published in ACM Computing Frontiers 2010 with on-chip testing, simulation results indicate the viability of SpiNNaker for large-scale neural modelling, while emphasizing the need for effective burst management and network mapping. Ultimately, the goal is the creation of a library-based development system that can translate a high-level neural model from any description environment into an efficient SpiNNaker instantiation. The complete system represents a general-purpose platform that can generate an arbitrary neural network and run it with hardware speed and scale.

[1]  Andres Upegui,et al.  An FPGA platform for on-line topology exploration of spiking neural networks , 2005, Microprocess. Microsystems.

[2]  John Wawrzynek,et al.  Silicon Auditory Processors as Computer Peripherals , 1992, NIPS.

[3]  Gert Cauwenberghs,et al.  Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses , 2007, IEEE Transactions on Neural Networks.

[4]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[5]  Professor Moshe Abeles,et al.  Local Cortical Circuits , 1982, Studies of Brain Function.

[6]  Ammar Belatreche,et al.  Challenges for large-scale implementations of spiking neural networks on FPGAs , 2007, Neurocomputing.

[7]  Alberto Prieto,et al.  VLSI Implementation of a Neural Model Using Spikes , 1997 .

[8]  Patrice Y. Simard,et al.  Using GPUs for machine learning algorithms , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[9]  Steve B. Furber,et al.  A universal abstract-time platform for real-time neural networks , 2009, 2009 International Joint Conference on Neural Networks.

[10]  Javier Navaridas,et al.  Simulating and evaluating interconnection networks with INSEE , 2011, Simul. Model. Pract. Theory.

[11]  Daniel Lehmann,et al.  A Model for Representing the Dynamics of a System of Synfire Chains , 2005, Journal of Computational Neuroscience.

[12]  Bing J. Sheu,et al.  General-purpose neural chips with electrically programmable synapses and gain-adjustable neurons , 1992 .

[13]  Heik Heinrich Hellmich,et al.  Synaptic plasticity in spiking neural networks (SP2INN): a system approach , 2003, IEEE Trans. Neural Networks.

[14]  William J. Dally,et al.  Deadlock-Free Message Routing in Multiprocessor Interconnection Networks , 1987, IEEE Transactions on Computers.

[15]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[16]  O. Sporns,et al.  Mapping the Structural Core of Human Cerebral Cortex , 2008, PLoS biology.

[17]  Qingxiang Wu,et al.  A Novel Approach for the Implementation of Large Scale Spiking Neural Networks on FPGA Hardware , 2005, IWANN.

[18]  Piotr Dudek,et al.  Reconfigurable platforms and the challenges for large-scale implementations of spiking neural networks , 2008, 2008 International Conference on Field Programmable Logic and Applications.

[19]  Dharmendra S. Modha,et al.  Anatomy of a cortical simulator , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[20]  Luis A. Plana,et al.  SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[21]  Steve B. Furber,et al.  Implementing Learning on the SpiNNaker Universal Neural Chip Multiprocessor , 2009, ICONIP.

[22]  S. Tam,et al.  An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses , 1990, International 1989 Joint Conference on Neural Networks.

[23]  G. Edelman,et al.  Large-scale model of mammalian thalamocortical systems , 2008, Proceedings of the National Academy of Sciences.

[24]  Steve B. Furber,et al.  Understanding the interconnection network of SpiNNaker , 2009, ICS.

[25]  Michael L. Hines,et al.  Just-in-Time Connectivity for Large Spiking Networks , 2008, Neural Computation.

[26]  Ulrich Rückert,et al.  ULSI Architectures for Artificial Neural Networks , 2001, IEEE Micro.

[27]  H. Graf,et al.  A CMOS associative memory chip based on neural networks , 1987, 1987 IEEE International Solid-State Circuits Conference. Digest of Technical Papers.

[28]  Pierre Yger,et al.  PyNN: A Common Interface for Neuronal Network Simulators , 2008, Front. Neuroinform..

[29]  Michael L. Hines,et al.  Parallel network simulations with NEURON , 2006, Journal of Computational Neuroscience.

[30]  Tarek M. Taha,et al.  A preliminary investigation of a neocortex model implementation on the Cray XD1 , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[31]  Gert Cauwenberghs,et al.  An analog VLSI recurrent neural network learning a continuous-time trajectory , 1996, IEEE Trans. Neural Networks.

[32]  Simon J Thorpe,et al.  SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons , 2003, Network.

[33]  Stephen B. Furber,et al.  Efficient modelling of spiking neural networks on a scalable chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[34]  Anthony Mouraud,et al.  A Distributed and Multithreaded Neural Event Driven Simulation Framework , 2006, Parallel and Distributed Computing and Networks.

[35]  Paolo Del Giudice,et al.  Efficient Event-Driven Simulation of Large Networks of Spiking Neurons and Dynamical Synapses , 2000, Neural Computation.

[36]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[37]  Luis A. Plana,et al.  A GALS Infrastructure for a Massively Parallel Multiprocessor , 2007, IEEE Design & Test of Computers.

[38]  Nikil D. Dutt,et al.  A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors , 2009, Neural Networks.

[39]  M. Abeles Local Cortical Circuits: An Electrophysiological Study , 1982 .

[40]  Eduardo Ros,et al.  Event-Driven Simulation Scheme for Spiking Neural Networks Using Lookup Tables to Characterize Neuronal Dynamics , 2006, Neural Computation.

[41]  Steve B. Furber,et al.  Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker , 2010, CF '10.

[42]  Cameron Patterson,et al.  Scalable event-driven native parallel processing: the SpiNNaker neuromimetic system , 2010, Conf. Computing Frontiers.

[43]  Steve B. Furber,et al.  The Leaky Integrate-and-Fire neuron: A platform for synaptic model exploration on the SpiNNaker chip , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[44]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[45]  Steve B. Furber,et al.  The Deferred Event Model for Hardware-Oriented Spiking Neural Networks , 2008, ICONIP.

[46]  Lloyd Watts,et al.  Event-Driven Simulation of Networks of Spiking Neurons , 1993, NIPS.

[47]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[48]  Ulrich Rückert,et al.  Implementation of artificial neural networks on a reconfigurable hardware accelerator , 2002, Proceedings 10th Euromicro Workshop on Parallel, Distributed and Network-based Processing.

[49]  Francisco Javier López Aligué,et al.  NeuSim: A Modular Neural Networks Simulator for Beowulf Clusters , 2001, IWANN.

[50]  Steve B. Furber,et al.  A General-Purpose Model Translation System for a Universal Neural Chip , 2010, ICONIP.

[51]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[52]  Javier Navaridas,et al.  SpiNNaker: impact of traffic locality, causality and burstiness on the performance of the interconnection network , 2010, Conf. Computing Frontiers.

[53]  M. Yagyu,et al.  Design, fabrication and evaluation of a 5-inch wafer scale neural network LSI composed on 576 digital neurons , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[54]  Mark James,et al.  Design of Low-Cost, Real-Time Simulation Systems for Large Neural Networks , 1992, J. Parallel Distributed Comput..

[55]  Gert Cauwenberghs,et al.  Analog VLSI Biophysical Neurons and Synapses With Programmable Membrane Channel Kinetics , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[56]  Anders Lansner,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[57]  Stephen B. Furber,et al.  Virtual synaptic interconnect using an asynchronous network-on-chip , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[58]  Romain Brette,et al.  Neuroinformatics Original Research Article Brian: a Simulator for Spiking Neural Networks in Python , 2022 .

[59]  Steve B. Furber,et al.  Optimal connectivity in hardware-targetted MLP networks , 2009, 2009 International Joint Conference on Neural Networks.

[60]  Johannes Schemmel,et al.  Realizing biological spiking network models in a configurable wafer-scale hardware system , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).