Event management for large scale event-driven digital hardware spiking neural networks

The interest in brain-like computation has led to the design of a plethora of innovative neuromorphic systems. Individually, spiking neural networks (SNNs), event-driven simulation and digital hardware neuromorphic systems get a lot of attention. Despite the popularity of event-driven SNNs in software, very few digital hardware architectures are found. This is because existing hardware solutions for event management scale badly with the number of events. This paper introduces the structured heap queue, a pipelined digital hardware data structure, and demonstrates its suitability for event management. The structured heap queue scales gracefully with the number of events, allowing the efficient implementation of large scale digital hardware event-driven SNNs. The scaling is linear for memory, logarithmic for logic resources and constant for processing time. The use of the structured heap queue is demonstrated on a field-programmable gate array (FPGA) with an image segmentation experiment and a SNN of 65,536 neurons and 513,184 synapses. Events can be processed at the rate of 1 every 7 clock cycles and a 406×158 pixel image is segmented in 200 ms.

[1]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[2]  Andrew S. Cassidy,et al.  Design of a one million neuron single FPGA neuromorphic system for real-time multimodal scene analysis , 2011, 2011 45th Annual Conference on Information Sciences and Systems.

[3]  Rodrigo Agís,et al.  Hardware event-driven simulation engine for spiking neural networks , 2007 .

[4]  Paolo Del Giudice,et al.  Efficient Event-Driven Simulation of Large Networks of Spiking Neurons and Dynamical Synapses , 2000, Neural Computation.

[5]  Eytan Domany,et al.  Temporal aspects of coding and information processing in biological systems , 1994 .

[6]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[7]  Jean Rouat,et al.  The oscillatory dynamic link matcher for spiking-neuron-based pattern recognition , 2006, Neurocomputing.

[8]  Tim Schönauer,et al.  NeuroPipe-Chip: A digital neuro-processor for spiking neural networks , 2002, IEEE Trans. Neural Networks.

[9]  Arindam Basu,et al.  Neural Dynamics in Reconfigurable Silicon , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[10]  Benjamin Schrauwen,et al.  Accelerating Event-Driven Simulation of Spiking Neurons with Multiple Synaptic Time Constants , 2009, Neural Computation.

[11]  Giacomo Indiveri,et al.  A VLSI network of spiking neurons with an asynchronous static random access memory , 2011, 2011 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[12]  Markus Diesmann,et al.  Advancing the Boundaries of High-Connectivity Network Simulation with Distributed Computing , 2005, Neural Computation.

[13]  Wayne Luk,et al.  FPGA Accelerated Simulation of Biologically Plausible Spiking Neural Networks , 2009, 2009 17th IEEE Symposium on Field Programmable Custom Computing Machines.

[14]  Jean Rouat,et al.  Monophonic sound source separation with an unsupervised network of spiking neurones , 2007, Neurocomputing.

[15]  Lászlo Bako.  Real-Time Clustering of Datasets with Hardware Embedded Neuromorphic Neural Networks , 2009, 2009 International Workshop on High Performance Computational Systems Biology.

[16]  Eduardo Ros,et al.  Real-time computing platform for spiking neurons (RT-spike) , 2006, IEEE Trans. Neural Networks.

[17]  Nikil D. Dutt,et al.  A configurable simulation environment for the efficient simulation of large-scale spiking neural networks on graphics processors , 2009, Neural Networks.

[18]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[19]  Bill Lin,et al.  Fast and scalable priority queue architecture for high-speed network switches , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[20]  P. Milner.  A model for visual shape recognition. , 1974, Psychological review.

[21]  Benjamin Schrauwen,et al.  Compact hardware liquid state machines on FPGA for real-time speech recognition , 2008, Neural Networks.

[22]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[23]  Lloyd Watts,et al.  Event-Driven Simulation of Networks of Spiking Neurons , 1993, NIPS.

[24]  Anthony Mouraud,et al.  Simulation of Large Spiking Neural Networks on Distributed Architectures, The "DAMNED" Simulator , 2009, EANN.

[25]  Manolis Katevenis,et al.  Pipelined Heap (Priority Queue) Management for Advanced Scheduling in High-Speed Networks , 2007, IEEE/ACM Transactions on Networking.

[26]  Wayne Luk,et al.  A Large-Scale Spiking Neural Network Accelerator for FPGA Systems , 2012, ICANN.

[27]  Simon J Thorpe,et al.  SpikeNET: an event-driven simulation package for modelling large networks of spiking neurons , 2003, Network.

[28]  Luis A. Plana,et al.  A GALS Infrastructure for a Massively Parallel Multiprocessor , 2007, IEEE Design & Test of Computers.

[29]  Heik Heinrich Hellmich,et al.  Synaptic plasticity in spiking neural networks (SP2INN): a system approach , 2003, IEEE Trans. Neural Networks.

[30]  Markus Diesmann,et al.  Exact Subthreshold Integration with Continuous Spike Times in Discrete-Time Neural Network Simulations , 2007, Neural Computation.

[31]  Joachim K. Anlauf,et al.  Fast Digital Simulation of Spiking Neural Networks and Neuromorphic Integration with Spikelab , 1999, Int. J. Neural Syst..

[32]  Liam McDaid,et al.  Hardware spiking neural network prototyping and application , 2011, Genetic Programming and Evolvable Machines.

[33]  Louis-Charles Caron,et al.  FPGA implementation of a spiking neural network for pattern matching , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[34]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[35]  Nabil H. Farhat,et al.  The double queue method: a numerical method for integrate-and-fire neuron networks , 2001, Neural Networks.

[36]  Gill A. Pratt.  Pulse computation , 1990 .