Optimizing Semantic Pointer Representations for Symbol-Like Processing in Spiking Neural Networks

The Semantic Pointer Architecture (SPA) is a proposal of specifying the computations and architectural elements needed to account for cognitive functions. By means of the Neural Engineering Framework (NEF) this proposal can be realized in a spiking neural network. However, in any such network each SPA transformation will accumulate noise. By increasing the accuracy of common SPA operations, the overall network performance can be increased considerably. As well, the representations in such networks present a trade-off between being able to represent all possible values and being only able to represent the most likely values, but with high accuracy. We derive a heuristic to find the near-optimal point in this trade-off. This allows us to improve the accuracy of common SPA operations by up to 25 times. Ultimately, it allows for a reduction of neuron number and a more efficient use of both traditional and neuromorphic hardware, which we demonstrate here.

[1]  Tony A. Plate,et al.  Holographic reduced representations , 1995, IEEE Trans. Neural Networks.

[2]  Ross W. Gayler.  Vector Symbolic Architectures answer Jackendoff's challenges for cognitive neuroscience , 2004, ArXiv.

[3]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[4]  Chris Eliasmith,et al.  A Controlled Attractor Network Model of Path Integration in the Rat , 2005, Journal of Computational Neuroscience.

[5]  C. Eliasmith.  Cognition with neurons: A large-scale, biologically realistic model of the Wason task , 2005 .

[6]  C. Anderson,et al.  A model of the computations leading to a representation of auditory space in the midbrain of the barn owl , 2005 .

[7]  Chris Eliasmith,et al.  Integrating behavioral and neural data in a model of zebrafish network interaction , 2005, Biological Cybernetics.

[8]  C. Eliasmith,et al.  Higher-Dimensional Neurons Explain the Tuning and Dynamics of Working Memory Cells , 2006, The Journal of Neuroscience.

[9]  M. Konishi,et al.  Emergence of multiplicative auditory responses in the midbrain of the barn owl. , 2007, Journal of neurophysiology.

[10]  Paul Thagard,et al.  Neural affective decision theory: Choices, brains, and emotions , 2008, Cognitive Systems Research.

[11]  Tobi Delbrück,et al.  CAVIAR: A 45k Neuron, 5M Synapse, 12G Connects/s AER Hardware Sensory–Processing– Learning–Actuating System for High-Speed Visual Object Recognition and Tracking , 2009, IEEE Transactions on Neural Networks.

[12]  Kwabena Boahen,et al.  Silicon Neurons That Compute , 2012, ICANN.

[13]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[14]  Chris Eliasmith,et al.  How to Build a Brain: A Neural Architecture for Biological Cognition , 2013 .

[15]  Chris Eliasmith,et al.  A spiking neural model applied to the study of human performance and cognitive decline on Raven's Advanced , 2014 .

[16]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[17]  Rodrigo Alvarez-Icaza,et al.  Neurogrid: A Mixed-Analog-Digital Multichip System for Large-Scale Neural Simulations , 2014, Proceedings of the IEEE.

[18]  M. Laubach,et al.  A Spiking Neural Integrator Model of the Adaptive Control of Action by the Medial Prefrontal Cortex , 2014, The Journal of Neuroscience.

[19]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[20]  Trevor Bekolay,et al.  Nengo: a Python tool for building large-scale functional brain models , 2014, Front. Neuroinform..

[21]  Terrence C. Stewart,et al.  Sentence processing in spiking neurons: A biologically plausible left-corner parser , 2014, CogSci.

[22]  Lambert Schomaker,et al.  Indoor localization by denoising autoencoders and semi-supervised learning in 3D simulated environment , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[23]  Stephen B. Furber,et al.  An efficient SpiNNaker implementation of the Neural Engineering Framework , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[24]  Chris Eliasmith,et al.  A Spiking Neural Model of the n-Back Task , 2015, CogSci.

[25]  André van Schaik,et al.  Neuromorphic Hardware Architecture Using the Neural Engineering Framework for Pattern Recognition , 2017, IEEE Transactions on Biomedical Circuits and Systems.