A Parallel Supercomputer Implementation of a Biological Inspired Neural Network and its use for Pattern Recognition

A parallel implementation of a large spiking neural network is proposed and evaluated. The neural network implements the binding by synchrony process using the Oscillatory Dynamic Link Matcher (ODLM). Scalability, speed and performance are compared for 2 implementations: Message Passing Interface (MPI) and Compute Unified Device Architecture (CUDA) running on clusters of multicore supercomputers and NVIDIA graphical processing units respectively. A global spiking list that represents at each instant the state of the neural network is described. This list indexes each neuron that fires during the current simulation time so that the influence of their spikes are simultaneously processed on all computing units. Our implementation shows a good scalability for very large networks. A complex and large spiking neural network has been implemented in parallel with success, thus paving the road towards real-life applications based on networks of spiking neurons. MPI offers a better scalability than CUDA, while the CUDA implementation on a GeForce GTX 285 gives the best cost to performance ratio. When running the neural network on the GTX 285, the processing speed is comparable to the MPI implementation on RQCHP's Mammouth parallel with 64 notes (128 cores).

[1]  Michel Dagenais,et al.  Internal Clock Drift Estimation in Computer Clusters , 2008, J. Comput. Networks Commun..

[2]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[3]  B. Scheuermann,et al.  On the Time Synchronization of Distributed Log Files in Networks With Local Broadcast Media , 2009, IEEE/ACM Transactions on Networking.

[4]  Jean-Charles Grégoire,et al.  Low-complexity offline and online clock skew estimation and removal , 2006, Comput. Networks.

[5]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[6]  Emiliano Betti,et al.  A global operating system for HPC clusters , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[7]  Nikil D. Dutt,et al.  Computing spike-based convolutions on GPUs , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[8]  Michel Dagenais,et al.  Accurate offline synchronization of distributed traces using kernel-level events , 2010, OPSR.

[9]  M. Desnoyers.  Low-Impact Operating System Tracing , 2009 .

[10]  Michael A. Lombardi,et al.  Time and Frequency From A to Z , 2002 .

[11]  Nicholas T. Carnevale,et al.  Simulation of networks of spiking neurons: A review of tools and strategies , 2006, Journal of Computational Neuroscience.

[12]  P. Milner.  A model for visual shape recognition. , 1974, Psychological review.

[13]  Renaud Sirdey,et al.  A linear programming approach to highly precise clock synchronization over a packet network , 2008, 4OR.

[14]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[15]  Jean Rouat,et al.  The oscillatory dynamic link matcher for spiking-neuron-based pattern recognition , 2006, Neurocomputing.

[16]  P. Ashton.  Algorithms For Off-line Clock Synchronization , 1995 .