Spiking Neural Classifier with Lumped Dendritic Nonlinearity and Binary Synapses: A Current Mode VLSI Implementation and Analysis

We present a neuromorphic current mode implementation of a spiking neural classifier with lumped square law dendritic nonlinearity. It has been shown previously in software simulations that such a system with binary synapses can be trained with structural plasticity algorithms to achieve comparable classification accuracy with fewer synaptic resources than conventional algorithms. We show that even in real analog systems with manufacturing imperfections (CV of 23.5% and 14.4% for dendritic branch gains and leaks respectively), this network is able to produce comparable results with fewer synaptic resources. The chip fabricated in m complementary metal oxide semiconductor has eight dendrites per cell and uses two opposing cells per class to cancel common-mode inputs. The chip can operate down to a V and dissipates 19 nW of static power per neuronal cell and 125 pJ/spike. For two-class classification problems of high-dimensional rate encoded binary patterns, the hardware achieves comparable performance as software implementation of the same with only about a 0.5% reduction in accuracy. On two UCI data sets, the IC integrated circuit has classification accuracy comparable to standard machine learners like support vector machines and extreme learning machines while using two to five times binary synapses. We also show that the system can operate on mean rate encoded spike patterns, as well as short bursts of spikes. To the best of our knowledge, this is the first attempt in hardware to perform classification exploiting dendritic properties and binary synapses.

[1]  Chiara Bartolozzi,et al.  Synaptic Dynamics in Analog VLSI , 2007, Neural Computation.

[2]  E. Culurciello,et al.  A biomorphic digital image sensor , 2003, IEEE J. Solid State Circuits.

[3]  Shaista Hussain,et al.  Morphological learning: Increased memory capacity of neuromorphic systems with binary synapses exploiting AER based reconfiguration , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[4]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[5]  Shaista Hussain,et al.  Hardware efficient, neuromorphic dendritically enhanced readout for liquid state machines , 2013, 2013 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[6]  Sundaram Suresh,et al.  Sequential Projection-Based Metacognitive Learning in a Radial Basis Function Network for Classification Problems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[7]  André van Schaik,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Scott Koziol,et al.  Modeling and Implementation of Voltage-Mode CMOS Dendrites on a Reconfigurable Analog Platform , 2012, IEEE Transactions on Biomedical Circuits and Systems.

[9]  H. Markram,et al.  Spontaneous and evoked synaptic rewiring in the neonatal neocortex , 2006, Proceedings of the National Academy of Sciences.

[10]  Shaista Hussain,et al.  Hardware-Amenable Structural Learning for Spike-Based Pattern Classification Using a Simple Model of Active Dendrites , 2014, Neural Computation.

[11]  Kwabena Boahen,et al.  Synchrony in Silicon: The Gamma Rhythm , 2007, IEEE Transactions on Neural Networks.

[12]  Bartlett W. Mel,et al.  Impact of Active Dendrites and Structural Plasticity on the Memory Capacity of Neural Tissue , 2001, Neuron.

[13]  Yingxue Wang,et al.  Active Processing of Spatio-Temporal Input Patterns in Silicon Dendrites , 2013, IEEE Transactions on Biomedical Circuits and Systems.

[14]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[15]  Subhrajit Roy,et al.  Liquid State Machine With Dendritically Enhanced Readout for Low-Power, Neuromorphic VLSI Implementations , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[16]  K. Svoboda,et al.  Long-term in vivo imaging of experience-dependent synaptic plasticity in adult cortex , 2002, Nature.

[17]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[18]  Kea-Tiong Tang,et al.  VLSI Implementation of a Bio-Inspired Olfactory Spiking Neural Network , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[19]  C. Gilbert,et al.  Axons and Synaptic Boutons Are Highly Dynamic in Adult Visual Cortex , 2006, Neuron.

[20]  Bartlett W. Mel,et al.  Computational subunits in thin dendrites of pyramidal cells , 2004, Nature Neuroscience.

[21]  Giacomo Indiveri,et al.  A low-power adaptive integrate-and-fire neuron circuit , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[22]  Shaista Hussain,et al.  Improved margin multi-class classification using dendritic neurons with morphological learning , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[23]  Kevan A. C. Martin,et al.  A Canonical Microcircuit for Neocortex , 1989, Neural Computation.

[24]  Subhrajit Roy,et al.  A current-mode spiking neural classifier with lumped dendritic nonlinearity , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[25]  Jennifer Hasler,et al.  Finding a roadmap to achieve large neuromorphic hardware systems , 2013, Front. Neurosci..

[26]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[27]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[28]  Richard George,et al.  Structural Plasticity Denoises Responses and Improves Learning Speed , 2016, Front. Comput. Neurosci..

[29]  Yingxue Wang,et al.  Multilayer Processing of Spatiotemporal Spike Patterns in a Neuron with Active Dendrites , 2010, Neural Computation.

[30]  Tobi Delbruck,et al.  Real-time classification and sensor fusion with a spiking deep belief network , 2013, Front. Neurosci..

[31]  Henry Markram,et al.  On the computational power of circuits of spiking neurons , 2004, J. Comput. Syst. Sci..

[32]  André van Schaik,et al.  A mixed-signal implementation of a polychronous spiking neural network with delay adaptation , 2014, Front. Neurosci..

[33]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[34]  Subhrajit Roy,et al.  Architectural exploration for on-chip, online learning in spiking neural networks , 2014, 2014 International Symposium on Integrated Circuits (ISIC).

[35]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[36]  Giacomo Indiveri,et al.  Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI , 2009, IEEE Transactions on Biomedical Circuits and Systems.

[37]  Gert Cauwenberghs,et al.  Event-driven contrastive divergence for spiking neuromorphic systems , 2013, Front. Neurosci..

[38]  Arindam Basu,et al.  A Learning-Enabled Neuron Array IC Based Upon Transistor Channel Models of Biological Phenomena , 2013, IEEE Transactions on Biomedical Circuits and Systems.

[39]  Shih-Chii Liu,et al.  Computation with Spikes in a Winner-Take-All Network , 2009, Neural Computation.