Turn Down That Noise: Synaptic Encoding of Afferent SNR in a Single Spiking Neuron

We have added a simplified neuromorphic model of Spike Time Dependent Plasticity (STDP) to the previously described Synapto-dendritic Kernel Adapting Neuron (SKAN), a hardware efficient neuron model capable of learning spatio-temporal spike patterns. The resulting neuron model is the first to perform synaptic encoding of afferent signal-to-noise ratio in addition to the unsupervised learning of spatio-temporal spike patterns. The neuron model is particularly suitable for implementation in digital neuromorphic hardware as it does not use any complex mathematical operations and uses a novel shift-based normalization approach to achieve synaptic homeostasis. The neuron's noise compensation properties are characterized and tested on random spatio-temporal spike patterns as well as a noise corrupted subset of the zero images of the MNIST handwritten digit dataset. Results show the simultaneously learning common patterns in its input data while dynamically weighing individual afferents based on their signal to noise ratio. Despite its simplicity the interesting behaviors of the neuron model and the resulting computational power may also offer insights into biological systems.

[1]  B. Achiriloaie,et al.  VI REFERENCES , 1961 .

[2]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[3]  E. Marder,et al.  Variability, compensation and homeostasis in neuron and network function , 2006, Nature Reviews Neuroscience.

[4]  T. Delbruck,et al.  A 128 128 120 dB 15 s Latency Asynchronous Temporal Contrast Vision Sensor , 2006 .

[5]  G. Tononi,et al.  Sleep function and synaptic homeostasis. , 2006, Sleep medicine reviews.

[6]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[7]  G. Turrigiano.  The Self-Tuning Neuron: Synaptic Scaling of Excitatory Synapses , 2008, Cell.

[8]  Giacomo Indiveri,et al.  Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI , 2009, IEEE Transactions on Biomedical Circuits and Systems.

[9]  Wulfram Gerstner,et al.  Spike-timing dependent plasticity , 2010, Scholarpedia.

[10]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[11]  Johannes Schemmel,et al.  Is a 4-Bit Synaptic Weight Resolution Enough? – Constraints on Enabling Spike-Timing Dependent Plasticity in Neuromorphic Hardware , 2012, Front. Neurosci..

[12]  W. Gerstner,et al.  Spike-Timing-Dependent Plasticity: A Comprehensive Overview , 2012, Front. Syn. Neurosci..

[13]  Omid Kavehei,et al.  Emergence of competitive control in a memristor-based neuromorphic circuit , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[14]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[15]  J. Tapson,et al.  The adaptation of spike backpropagation delays in cortical neurons , 2013, Front. Cell. Neurosci..

[16]  Gregory Cohen,et al.  The ripple pond: enabling spiking networks to see , 2013, Front. Neurosci..

[17]  Liam McDaid,et al.  A simple programmable axonal delay scheme for spiking neural networks , 2013, Neurocomputing.

[18]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[19]  André van Schaik,et al.  A compact reconfigurable mixed-signal implementation of synaptic plasticity in spiking neurons , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[20]  Mostafa Rahimi Azghadi,et al.  Spike-Based Synaptic Plasticity in Silicon: Design, Implementation, Application, and Challenges , 2014, Proceedings of the IEEE.

[21]  Tara Julia Hamilton,et al.  The Synaptic Kernel Adaptation Network , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[22]  Shaista Hussain,et al.  Delay learning architectures for memory and classification , 2014, Neurocomputing.

[23]  André van Schaik,et al.  Stochastic Electronics: A Neuro-Inspired Design Paradigm for Integrated Circuits , 2014, Proceedings of the IEEE.

[24]  André van Schaik,et al.  Racing to learn: statistical inference and learning in a single spiking neuron with adaptive kernels , 2014, Front. Neurosci..

[25]  Shaista Hussain,et al.  Spike-timing dependent morphological learning for a neuron with nonlinear active dendrites , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[26]  Tara Julia Hamilton,et al.  Rotationally invariant vision recognition with neuromorphic transformation and learning networks , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[27]  André van Schaik,et al.  A mixed-signal implementation of a polychronous spiking neural network with delay adaptation , 2014, Front. Neurosci..

[28]  Bernabé Linares-Barranco,et al.  Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network , 2015, IEEE Transactions on Neural Networks and Learning Systems.