Minimal Spiking Neuron for Solving Multilabel Classification Tasks

The multispike tempotron (MST) is a powersul, single spiking neuron model that can solve complex supervised classification tasks. It is also internally complex, computationally expensive to evaluate, and unsuitable for neuromorphic hardware. Here we aim to understand whether it is possible to simplify the MST model while retaining its ability to learn and process information. To this end, we introduce a family of generalized neuron models (GNMs) that are a special case of the spike response model and much simpler and cheaper to simulate than the MST. We find that over a wide range of parameters, the GNM can learn at least as well as the MST does. We identify the temporal autocorrelation of the membrane potential as the most important ingredient of the GNM that enables it to classify multiple spatiotemporal patterns. We also interpret the GNM as a chemical system, thus conceptually bridging computation by neural networks with molecular information processing. We conclude the letter by proposing alternative training approaches for the GNM, including error trace learning and error backpropagation.

[1]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[2]  Jim D. Garside,et al.  SpiNNaker: Design and Implementation of a GALS Multicore System-on-Chip , 2011, JETC.

[3]  B. Rajendran,et al.  Neuromorphic Computing Based on Emerging Memory Technologies , 2016, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[4]  D. Feldman The Spike-Timing Dependence of Plasticity , 2012, Neuron.

[5]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[6]  Wulfram Gerstner,et al.  Mathematical formulations of Hebbian learning , 2002, Biological Cybernetics.

[7]  Stefan Schliebs,et al.  Span: Spike Pattern Association Neuron for Learning Spatio-Temporal Spike Patterns , 2012, Int. J. Neural Syst..

[8]  Razvan V. Florian,et al.  The Chronotron: A Neuron That Learns to Fire Temporally Precise Spike Patterns , 2010, PloS one.

[9]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[10]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[11]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[12]  Emre Neftci,et al.  Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks , 2019, IEEE Signal Processing Magazine.

[13]  Kaiming He,et al.  Exploring the Limits of Weakly Supervised Pretraining , 2018, ECCV.

[14]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[15]  Rémi Monasson,et al.  Theory of spike timing-based neural classifiers. , 2010, Physical review letters.

[16]  Robert Gütig,et al.  Spiking neurons can discover predictive features by aggregate-label learning , 2016, Science.

[17]  Wulfram Gerstner,et al.  Reducing Spike Train Variability: A Computational Theory Of Spike-Timing Dependent Plasticity , 2004, BNAIC.

[18]  WangHong,et al.  Mapping spiking neural networks onto a manycore neuromorphic architecture , 2018 .

[19]  Huajin Tang,et al.  Precise-Spike-Driven Synaptic Plasticity: Learning Hetero-Association of Spatiotemporal Spike Patterns , 2013, PloS one.

[20]  Nicolas Brunel,et al.  Lapicque’s 1907 paper: from frogs to integrate-and-fire , 2007, Biological Cybernetics.

[21]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[22]  Hong Wang,et al.  Mapping spiking neural networks onto a manycore neuromorphic architecture , 2018, PLDI.

[23]  Pierre Boulet,et al.  Spiking Neural Computing in Memristive Neuromorphic Platforms , 2019, Handbook of Memristor Networks.

[24]  Johannes Schemmel,et al.  Demonstrating Advantages of Neuromorphic Computation: A Pilot Study , 2018, Front. Neurosci..

[25]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[26]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[27]  Anthony Maida,et al.  BP-STDP: Approximating Backpropagation using Spike Timing Dependent Plasticity , 2017, Neurocomputing.

[28]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[29]  F. Ponulak ReSuMe-New Supervised Learning Method for Spiking Neural Networks , 2005 .