Efficient modelling of spiking neural networks on a scalable chip multiprocessor

We propose a system based on the Izhikevich model running on a scalable chip multiprocessor - SpiNNaker - for large-scale spiking neural network simulation. The design takes into account the requirements for processing, storage, and communication which are essential to the efficient modelling of spiking neural networks. To gain a speedup of the processing as well as saving storage space, the Izhikevich model is implemented in 16-bit fixed-point arithmetic. An approach based on using two scaling factors is developed, making the precision comparable to the original. With the two scaling factors scheme, all of the firing patterns by the original model can be reproduced with a much faster execution speed. To reduce the communication overhead, rather than sending synaptic weights on communicating, we only send out event packets to indicate the neuron firings while holding the synaptic weights in the memory of the post-synaptic neurons, which is so-called event-driven algorithm. The communication based on event packets can be handled efficiently by the multicast system supported by the SpiNNaker machine. We also describe a system level model for spiking neural network simulation based on the schemes above. The model has been functionally verified and experimental results are included. An analysis of the performance of the whole system is presented at the end of the paper.

[1]  Stephen B. Furber,et al.  Chain: A Delay-Insensitive Chip Area Interconnect , 2002, IEEE Micro.

[2]  T. Schoenauer,et al.  MASPINN: novel concepts for a neuroaccelerator for spiking neural networks , 1999, Other Conferences.

[3]  Luis A. Plana,et al.  SpiNNaker: Mapping neural networks onto a massively-parallel chip multiprocessor , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[4]  Anthony G. Pipe,et al.  Design and FPGA implementation of an embedded real-time biologically plausible spiking neural network processor , 2005, International Conference on Field Programmable Logic and Applications, 2005..

[5]  Andrew D. Brown,et al.  On-chip and inter-chip networks for modeling large-scale neural systems , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[6]  Stephen B. Furber,et al.  Virtual synaptic interconnect using an asynchronous network-on-chip , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[7]  U. Ruckert,et al.  ParSPIKE-a parallel DSP-accelerator for dynamic simulation of large spiking neural networks , 1999, Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems.

[8]  Narasimhan Sundararajan,et al.  Parallel implementation of backpropagation neural networks on a heterogeneous array of transputers , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[9]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[10]  Anthony G. Pipe,et al.  Implementing Spiking Neural Networks for Real-Time Signal-Processing and Control Applications: A Model-Validated FPGA Approach , 2007, IEEE Transactions on Neural Networks.

[11]  Wolfgang Maass,et al.  Noisy Spiking Neurons with Temporal Coding have more Computational Power than Sigmoidal Neurons , 1996, NIPS.

[12]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[13]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[14]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[15]  Steve Furber,et al.  High-performance computing for systems of spiking neurons , 2006 .

[16]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[17]  Anthony G. Pipe,et al.  A Real-Time, FPGA Based, Biologically Plausible Neural Network Processor , 2005, ICANN.

[18]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[19]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.