Delay learning architectures for memory and classification

Abstract We present a neuromorphic spiking neural network, the DELTRON, that can remember and store patterns by changing the delays of every connection as opposed to modifying the weights. The advantage of this architecture over traditional weight-based ones is simpler hardware implementation without multipliers or digital–analog converters (DACs) as well as being suited to time-based computing. The name is derived due to similarity in the learning rule with an earlier architecture called tempotron. The DELTRON can remember more patterns than other delay-based networks by modifying a few delays to remember the most ‘salient’ or synchronous part of every spike pattern. We present simulations of memory capacity and classification ability of the DELTRON for different random spatio-temporal spike patterns. The memory capacity for noisy spike patterns and missing spikes is also shown. Finally, we present SPICE simulation results of the core circuits involved in a reconfigurable mixed signal implementation of this architecture.

[1]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[2]  Antoni Morro,et al.  Hardware Implementation of Stochastic Spiking Neural Networks , 2012, Int. J. Neural Syst..

[3]  André van Schaik,et al.  An aVLSI programmable axonal delay circuit with spike timing dependent delay adaptation , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[4]  Arindam Basu,et al.  Transistor channel dendrites implementing HMM classifiers , 2007, 2007 IEEE International Symposium on Circuits and Systems.

[5]  André van Schaik,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[6]  Luping Shi,et al.  Presynaptic Learning and Memory with a Persistent Firing Neuron and a Habituating Synapse: a Model of Short Term Persistent Habituation , 2012, Int. J. Neural Syst..

[7]  Giacomo Indiveri,et al.  A current-mode conductance-based silicon neuron for address-event neuromorphic systems , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[8]  R. Eckmiller,et al.  Information processing in biology-inspired pulse coded neural networks , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[9]  Chiara Bartolozzi,et al.  Synaptic Dynamics in Analog VLSI , 2007, Neural Computation.

[10]  Alexander H. Waibel,et al.  The Tempo 2 Algorithm: Adjusting Time-Delays By Supervised Learning , 1990, NIPS.

[11]  Michael Jansen,et al.  Recognition and tracking of impulse patterns with delay adaptation in biology-inspired pulse processing neural net (BPN) hardware , 2004, Biological Cybernetics.

[12]  Amine Bermak,et al.  Arbitrated Time-to-First Spike CMOS Image Sensor With On-Chip Histogram Equalization , 2007, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[13]  Arindam Basu,et al.  A Learning-Enabled Neuron Array IC Based Upon Transistor Channel Models of Biological Phenomena , 2013, IEEE Transactions on Biomedical Circuits and Systems.

[14]  Shaista Hussain,et al.  DELTRON: Neuromorphic architectures for delay based learning , 2012, 2012 IEEE Asia Pacific Conference on Circuits and Systems.

[15]  Wai Keung Wong,et al.  Relationship between Applicability of Current-Based Synapses and Uniformity of Firing Patterns , 2012, Int. J. Neural Syst..

[16]  John G. Harris,et al.  A time-to-first spike CMOS imager , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[17]  Günther Palm,et al.  Synaptic Delay Learning in Pulse-Coupled Neurons , 1998, Neural Computation.

[18]  Eugene M. Izhikevich,et al.  Polychronization: Computation with Spikes , 2006, Neural Computation.

[19]  Arindam Basu,et al.  Nullcline-Based Design of a Silicon Neuron , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[20]  Craig T. Jin,et al.  A programmable axonal propagation delay circuit for time-delay spiking neural networks , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[21]  Misha Anne Mahowald,et al.  VLSI analogs of neuronal visual processing: a synthesis of form and function , 1992 .

[22]  Amir F. Atiya,et al.  How delays affect neural dynamics and learning , 1994, IEEE Trans. Neural Networks.

[23]  Arindam Basu,et al.  Neural dynamics in reconfigurable silicon , 2010, ISCAS.

[24]  Arindam Basu,et al.  Bifurcations in a silicon neuron , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[25]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[26]  John J. Hopfield,et al.  Connected-digit speaker-dependent speech recognition using a neural network with time-delayed connections , 1991, IEEE Trans. Signal Process..

[27]  J. Harris,et al.  A Time-to-First-Spike CMOS Image Sensor , 2007 .

[28]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[29]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[30]  Hongming Zhou,et al.  Silicon spiking neurons for hardware implementation of extreme learning machines , 2013, Neurocomputing.

[31]  J J Hopfield,et al.  Neural computation by concentrating information in time. , 1987, Proceedings of the National Academy of Sciences of the United States of America.