Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition

A neuromorphic chip that combines CMOS analog spiking neurons and memristive synapses offers a promising solution to brain-inspired computing, as it can provide massive neural network parallelism and density. Previous hybrid analog CMOS-memristor approaches required extensive CMOS circuitry for training, and thus eliminated most of the density advantages gained by the adoption of memristor synapses. Further, they used different waveforms for pre and post-synaptic spikes that added undesirable circuit overhead. Here we describe a hardware architecture that can feature a large number of memristor synapses to learn real-world patterns. We present a versatile CMOS neuron that combines integrate-and-fire behavior, drives passive memristors and implements competitive learning in a compact circuit module, and enables in situ plasticity in the memristor synapses. We demonstrate handwritten-digits recognition using the proposed architecture using transistor-level circuit simulations. As the described neuromorphic architecture is homogeneous, it realizes a fundamental building block for large-scale energy-efficient brain-inspired silicon chips that could lead to next-generation cognitive computing.

[1]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[2]  Pinaki Mazumder,et al.  CMOS and Memristor-Based Neural Network Design for Position Detection , 2012, Proceedings of the IEEE.

[3]  Fabien Alibart,et al.  Plasticity in memristive devices for spiking neural networks , 2015, Front. Neurosci..

[4]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[5]  Khaled N. Salama,et al.  Memristor-based memory: The sneak paths problem and solutions , 2013, Microelectron. J..

[6]  Shimeng Yu,et al.  Design considerations of synaptic device for neuromorphic computing , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[7]  T. Serrano-Gotarredona,et al.  A Proposal for Hybrid Memristor-CMOS Spiking Neuromorphic Learning Systems , 2013, IEEE Circuits and Systems Magazine.

[8]  Jacques-Olivier Klein,et al.  Bioinspired networks with nanoscale memristive devices that combine the unsupervised and supervised learning approaches , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[9]  Sylvain Saïghi,et al.  Excitatory and Inhibitory Memristive Synapses for Spiking Neural Networks , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[10]  Steve B. Furber,et al.  Power analysis of large-scale, real-time neural networks on SpiNNaker , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[11]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[12]  André van Schaik,et al.  A generalised conductance-based silicon neuron for large-scale spiking neural networks , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[13]  Y. Dan,et al.  Spike timing-dependent plasticity: from synapse to perception. , 2006, Physiological reviews.

[14]  W. Lu,et al.  CMOS compatible nanoscale nonvolatile resistance switching memory. , 2008, Nano letters.

[15]  Yang Liu,et al.  Review of Nanostructured Resistive Switching Memristor and Its Applications , 2014 .

[16]  Narayan Srinivasa,et al.  Energy-Efficient Neuron, Synapse and STDP Integrated Circuits , 2012, IEEE Transactions on Biomedical Circuits and Systems.

[17]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[18]  F. Corinto,et al.  Memristor Model Comparison , 2013, IEEE Circuits and Systems Magazine.

[19]  Julie Grollier,et al.  Silicon neuron dedicated to memristive spiking neural networks , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[20]  Nace L. Golding,et al.  Dendritic spikes as a mechanism for cooperative long-term potentiation , 2002, Nature.

[21]  Byoung Hun Lee,et al.  Neuromorphic Hardware System for Visual Pattern Recognition With Memristor Array and CMOS Neuron , 2015, IEEE Transactions on Industrial Electronics.

[22]  Andrew S. Cassidy,et al.  A combinational digital logic approach to STDP , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[23]  Robert Legenstein,et al.  A compound memristive synapse model for statistical learning through STDP in spiking neural networks , 2014, Front. Neurosci..

[24]  Ron Hogervorst,et al.  A compact power-efficient 3 V CMOS rail-to-rail input/output operational amplifier for VLSI cell libraries , 1994 .

[25]  Bernabé Linares-Barranco,et al.  On Spike-Timing-Dependent-Plasticity, Memristive Devices, and Building a Self-Learning Visual Cortex , 2011, Front. Neurosci..

[26]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[27]  Wei D. Lu,et al.  Pattern recognition with memristor networks , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[28]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[29]  Chris Yakopcic,et al.  Generalized Memristive Device SPICE Model and its Application in Circuit Design , 2013, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[30]  H. Hwang,et al.  Hardware implementation of associative memory characteristics with analogue-type resistive-switching device , 2014, Nanotechnology.

[31]  Hongming Zhou,et al.  Silicon spiking neurons for hardware implementation of extreme learning machines , 2013, Neurocomputing.

[32]  Anton J. M. Loonen,et al.  Circuits regulating pleasure and happiness: the evolution of reward-seeking and misery-fleeing behavioral mechanisms in vertebrates , 2015, Front. Neurosci..

[33]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[34]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[35]  Shigeo Sato,et al.  Izhikevich neuron circuit using stochastic logic , 2014 .

[36]  X. Miao,et al.  Ultrafast Synaptic Events in a Chalcogenide Memristor , 2013, Scientific Reports.

[37]  D. Querlioz,et al.  Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices , 2013, IEEE Transactions on Nanotechnology.

[38]  Rainer Waser,et al.  Complementary resistive switches for passive nanocrossbar memories. , 2010, Nature materials.

[39]  Yiran Chen,et al.  Memristor Crossbar-Based Neuromorphic Computing System: A Case Study , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[41]  Y. Dan,et al.  Spike Timing-Dependent Plasticity of Neural Circuits , 2004, Neuron.

[42]  Giacomo Indiveri,et al.  Integration of nanoscale memristor synapses in neuromorphic computing architectures , 2013, Nanotechnology.

[43]  Huajin Tang,et al.  Precise-Spike-Driven Synaptic Plasticity: Learning Hetero-Association of Spatiotemporal Spike Patterns , 2013, PloS one.

[44]  Makoto Motoyoshi,et al.  Through-Silicon Via (TSV) , 2009, Proceedings of the IEEE.

[45]  Garrett S. Rose,et al.  Memristor-Based Neural Logic Blocks for Nonlinearly Separable Functions , 2013, IEEE Transactions on Computers.

[46]  Vivien A. Casagrande,et al.  Biophysics of Computation: Information Processing in Single Neurons , 1999 .

[47]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.

[48]  Yiran Chen,et al.  Memristor crossbar-based unsupervised image learning , 2013, Neural Computing and Applications.

[49]  Leon O. Chua,et al.  Adaptive Neuromorphic Architecture (ANA) , 2013, Neural Networks.

[50]  Bernabé Linares-Barranco,et al.  Memristance can explain Spike-Time-Dependent-Plasticity in Neural Synapses , 2009 .

[51]  J. Yang,et al.  Anatomy of a Nanoscale Conduction Channel Reveals the Mechanism of a High‐Performance Memristor , 2011, Advanced materials.

[52]  Zhaohao Wang,et al.  On-chip supervised learning rule for ultra high density neural crossbar using memristor for synapse and neuron , 2014, 2014 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[53]  Yuchao Yang,et al.  Building Neuromorphic Circuits with Memristive Devices , 2013, IEEE Circuits and Systems Magazine.

[54]  Hyunsang Hwang,et al.  Neuromorphic Character Recognition System With Two PCMO Memristors as a Synapse , 2014, IEEE Transactions on Industrial Electronics.

[55]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[56]  Damien Querlioz,et al.  Simulation of a memristor-based spiking neural network immune to device variations , 2011, The 2011 International Joint Conference on Neural Networks.

[57]  Tarek M. Taha,et al.  Enabling back propagation training of memristor crossbar neuromorphic processors , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[58]  O. Richard,et al.  10×10nm2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation , 2011, 2011 International Electron Devices Meeting.

[59]  David Moore,et al.  Silver chalcogenide based memristor devices , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[60]  H. Hwang,et al.  Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device , 2011, Nanotechnology.

[61]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[62]  T. Serrano-Gotarredona,et al.  Design of adaptive nano/CMOS neural architectures , 2012, 2012 19th IEEE International Conference on Electronics, Circuits, and Systems (ICECS 2012).

[63]  Shimeng Yu,et al.  Orientation classification by a winner-take-all network with oxide RRAM based synaptic devices , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[64]  Jiantao Zhou,et al.  Stochastic Memristive Devices for Computing and Neuromorphic Applications , 2013, Nanoscale.

[65]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[66]  Shimeng Yu,et al.  An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation , 2011, IEEE Transactions on Electron Devices.

[67]  Melika Payvand,et al.  A CMOS-memristive self-learning neural network for pattern classification applications , 2014, 2014 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[68]  Antoine Joubert,et al.  A robust and compact 65 nm LIF analog neuron for computational purposes , 2011, 2011 IEEE 9th International New Circuits and systems conference.

[69]  G. Snider,et al.  Self-organized computation with unreliable, memristive nanodevices , 2007 .

[70]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[71]  Heiko Neumann,et al.  Unsupervised Learning of Head Pose through Spike-Timing Dependent Plasticity , 2008, PIT.

[72]  Shimeng Yu,et al.  A Low Energy Oxide‐Based Electronic Synaptic Device for Neuromorphic Visual Systems with Tolerance to Device Variation , 2013, Advanced materials.

[73]  Narayan Srinivasa,et al.  A scalable neural chip with synaptic electronics using CMOS integrated memristors , 2013, Nanotechnology.

[74]  Damien Querlioz,et al.  Stochastic neuron design using conductive bridge RAM , 2013, 2013 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[75]  T Tyrrell,et al.  Cerebellar cortex: its simulation and the relevance of Marr's theory. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[76]  J. Yang,et al.  Memristive switching mechanism for metal/oxide/metal nanodevices. , 2008, Nature nanotechnology.