Simulation of bayesian learning and inference on distributed stochastic spiking neural networks

The ability of neural networks to perform pattern recognition, classification and associative memory, is essential to applications such as image and speech recognition, natural language understanding, decision making etc. In spiking neural networks (SNNs), information is encoded as sparsely distributed train of spikes, which allows learning through the spike-timing dependent plasticity (STDP) property. SNNs can potentially achieve very large scale implementation and distributed learning due to the inherent asynchronous and sparse inter-neuron communications. In this work, we develop an efficient, scalable and flexible SNN simulator, which supports learning through STDP. The simulator is ideal for biologically inspired neuron models for computation but not for biologically realistic models. Bayesian neuron model for SNNs that is capable of online and fully-distributed STDP learning is introduced. The function of the simulator is validated using two networks representing two different applications from unsupervised feature extraction to inference based sentence construction.

[1]  Romain Brette,et al.  Neuroinformatics Original Research Article Brian: a Simulator for Spiking Neural Networks in Python , 2022 .

[2]  Nicholas T. Carnevale,et al.  The NEURON Simulation Environment , 1997, Neural Computation.

[3]  James M. Bower,et al.  The book of GENESIS - exploring realistic neural models with the GEneral NEural SImulation System (2. ed.) , 1994 .

[4]  N. Ellouze,et al.  Self-organization map of spiking neurons evaluation in phoneme classification , 2012, 2012 6th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT).

[5]  B J Richmond,et al.  Stochastic nature of precisely timed spike patterns in visual system neuronal responses. , 1999, Journal of neurophysiology.

[6]  Sophie Denève,et al.  Bayesian Inference with Spiking Neurons , 2004, Encyclopedia of Computational Neuroscience.

[7]  Rajesh P. N. Rao,et al.  Bayesian brain : probabilistic approaches to neural coding , 2006 .

[8]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[9]  Lyle N. Long,et al.  Character Recognition using Spiking Neural Networks , 2007, 2007 International Joint Conference on Neural Networks.

[10]  Tobi Delbruck,et al.  Real-time classification and sensor fusion with a spiking deep belief network , 2013, Front. Neurosci..

[11]  Qing Wu,et al.  A Parallel Neuromorphic Text Recognition System and Its Implementation on a Heterogeneous High-Performance Computing Cluster , 2013, IEEE Transactions on Computers.

[12]  Rajesh P. N. Rao,et al.  Probabilistic Models of the Brain: Perception and Neural Function , 2002 .

[13]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[14]  Nikil D. Dutt,et al.  Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule , 2013, Neural Networks.

[15]  H. Seung,et al.  Learning in Spiking Neural Networks by Reinforcement of Stochastic Synaptic Transmission , 2003, Neuron.

[16]  Wulfram Gerstner,et al.  Spike-timing dependent plasticity , 2010, Scholarpedia.

[17]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[18]  David J. Freedman,et al.  Choice-correlated activity fluctuations underlie learning of neuronal category representation , 2015, Nature Communications.

[19]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[20]  Eduardo Ros,et al.  Event-Driven Simulation Scheme for Spiking Neural Networks Using Lookup Tables to Characterize Neuronal Dynamics , 2006, Neural Computation.

[21]  Jim D. Garside,et al.  Overview of the SpiNNaker System Architecture , 2013, IEEE Transactions on Computers.

[22]  Yu Wang,et al.  Energy Efficient RRAM Spiking Neural Network for Real Time Classification , 2015, ACM Great Lakes Symposium on VLSI.

[23]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.