Efficient SpiNNaker simulation of a heteroassociative memory using the Neural Engineering Framework

The biological brain is a highly plastic system within which the efficacy and structure of synaptic connections are constantly changing in response to internal and external stimuli. While numerous models of this plastic behavior exist at various levels of abstraction, how these mechanisms allow the brain to learn meaningful values is unclear. The Neural Engineering Framework (NEF) is a hypothesis about how large-scale neural systems represent values using populations of spiking neurons, and transform them using functions implemented by the synaptic weights between populations. By exploiting the fact that these connection weight matrices are factorable, we have recently shown that static NEF models can be simulated very efficiently using the SpiNNaker neuromorphic architecture. In this paper, we demonstrate how this approach can be extended to efficiently support both supervised and unsupervised learning rules designed to operate on these factored matrices. We then present a heteroassociative memory architecture built using these learning rules and prove that it is capable of learning a human-scale semantic network. Finally we demonstrate a 100 000 neuron version of this architecture running on the SpiNNaker simulator with a speed-up exceeding 150x when compared to the Nengo reference simulator.

[1]  Arnold J. Stromberg,et al.  Number-theoretic Methods in Statistics , 1996 .

[2]  Trevor Bekolay,et al.  Simultaneous unsupervised and supervised learning of cognitive functions in biologically plausible spiking neural networks , 2013, CogSci.

[3]  I. Sobol.  On the distribution of points in a cube and the approximate evaluation of integrals , 1967 .

[4]  Steve B. Furber,et al.  Power analysis of large-scale, real-time neural networks on SpiNNaker , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[5]  Trevor Bekolay,et al.  Nengo: a Python tool for building large-scale functional brain models , 2014, Front. Neuroinform..

[6]  Steve B. Furber,et al.  A framework for plasticity implementation on the SpiNNaker neural architecture , 2015, Front. Neurosci..

[7]  Eugene M. Izhikevich,et al.  Relating STDP to BCM , 2003, Neural Computation.

[8]  R. Quiroga.  Concept cells: the building blocks of declarative memory functions , 2012, Nature Reviews Neuroscience.

[9]  Steve B. Furber,et al.  Correctness and performance of the SpiNNaker architecture , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[10]  C. Eliasmith,et al.  Learning to Select Actions with Spiking Neurons in the Basal Ganglia , 2012, Front. Neurosci..

[11]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[12]  Aaron R. Voelker,et al.  Learning large-scale heteroassociative memories in spiking neurons , 2014 .

[13]  Chris Eliasmith,et al.  Fine-Tuning and the Stability of Recurrent Neural Networks , 2011, PloS one.

[14]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[15]  Giorgio A. Ascoli,et al.  A Neural Mechanism for Background Information-Gated Learning Based on Axonal-Dendritic Overlaps , 2015, PLoS Comput. Biol..

[16]  Aaron Russell Voelker.  A Solution to the Dynamics of the Prescribed Error Sensitivity Learning Rule , 2015 .

[17]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[18]  Stephen B. Furber,et al.  An efficient SpiNNaker implementation of the Neural Engineering Framework , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[19]  Sylvain Chartier,et al.  2009 Special Issue: A new bidirectional heteroassociative memory encompassing correlational, competitive and topological properties , 2009 .

[20]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[21]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[22]  Steve B. Furber,et al.  Event-Driven Simulation of Arbitrary Spiking Neural Networks on SpiNNaker , 2011, ICONIP.

[23]  W. Fischer,et al.  Sphere Packings, Lattices and Groups , 1990 .

[24]  Matthew Cook,et al.  Efficient implementation of STDP rules on SpiNNaker neuromorphic hardware , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[25]  Oliver Trujillo,et al.  A Spiking Neural Model of Episodic Memory Encoding and Replay in Hippocampus , 2014 .

[26]  Chris Eliasmith,et al.  Biologically Plausible, Human-scale Knowledge Representation , 2016, CogSci.

[27]  Steve B. Furber,et al.  Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[28]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..