Towards spiking neuromorphic system-on-a-chip with bio-plausible synapses using emerging devices

Large-scale integration of CMOS mixed-signal integrated circuits and nanoscale emerging memory devices, such as the resistive RAM (RRAM) crosspoint arrays, can enable a new generation of Neuromorphic computers that can alleviate the von Neumann bottleneck, and be applied to a wide range of cognitive computing tasks1. Such hybrid Neuromorphic System-on-a-chip (NeuSoC) architectures will result in deep learning capability at chip-scale form factors, and several orders of magnitude reduction in energy consumption. Progress in this area has been impeded as the performance of these emerging devices falls short of the expected behavior from the idealized analog synapses, or weights, and new learning algorithms are needed to take advantage of the devices. To address these, we present a pathway to realize spike-based mixed-signal neuromorphic architectures; bottom-up from device arrays, circuit motifs, to semi-supervised algorithms that can realize large scale deep learning, autonomous control, sensor fusion and inference systems with 'brain-like' energy-efficiency.

[1]  Shimeng Yu,et al.  Design considerations of synaptic device for neuromorphic computing , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[2]  Kehan Zhu,et al.  A CMOS spiking neuron for dense memristor-synapse connectivity for brain-inspired computing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[3]  M. R. Latif,et al.  Nano-Ionic Redox Resistive RAM – Device Performance Enhancement Through Materials Engineering, Characterization and Electrical Testing , 2014 .

[4]  R. Jordan,et al.  NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[5]  Vishal Saxena,et al.  A CMOS Spiking Neuron for Brain-Inspired Neural Networks With Resistive Synapses and In Situ Learning , 2015, IEEE Transactions on Circuits and Systems II: Express Briefs.

[6]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[7]  Y. Dan,et al.  Spike Timing-Dependent Plasticity of Neural Circuits , 2004, Neuron.

[8]  Giacomo Indiveri,et al.  Integration of nanoscale memristor synapses in neuromorphic computing architectures , 2013, Nanotechnology.

[9]  Vishal Saxena,et al.  Homogeneous Spiking Neuromorphic System for Real-World Pattern Recognition , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[10]  X. Miao,et al.  Ultrafast Synaptic Events in a Chalcogenide Memristor , 2013, Scientific Reports.

[11]  L. Goux,et al.  Understanding of the endurance failure in scaled HfO2-based 1T1R RRAM through vacancy mobility degradation , 2012, 2012 International Electron Devices Meeting.

[12]  Kwabena Boahen.  NEUROGRID: Emulating a million neurons in the cortex , 2010, Neuroscience Research.

[13]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[14]  O. Richard,et al.  10×10nm2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation , 2011, 2011 International Electron Devices Meeting.

[15]  H. Hwang,et al.  Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device , 2011, Nanotechnology.

[16]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[17]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.

[18]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[19]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[20]  Yuchao Yang,et al.  Building Neuromorphic Circuits with Memristive Devices , 2013, IEEE Circuits and Systems Magazine.

[21]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[22]  Wulfram Gerstner,et al.  Spike-timing dependent plasticity , 2010, Scholarpedia.

[23]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[24]  Anthony S. Maida,et al.  Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning , 2016, ArXiv.

[25]  Paul E. Hasler,et al.  Computing with networks of spiking neurons on a biophysically motivated floating-gate based neuromorphic integrated circuit , 2013, Neural Networks.

[26]  N. Cady,et al.  Nanoscale Hafnium Oxide RRAM Devices Exhibit Pulse Dependent Behavior and Multi-level Resistance Capability , 2016 .

[27]  C. Clopath,et al.  The Role of Neuromodulators in Cortical Plasticity. A Computational Perspective , 2017, Front. Synaptic Neurosci..

[28]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[29]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[30]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[31]  Vishal Saxena,et al.  Enabling bio-plausible multi-level STDP using CMOS neurons with dendrites and bistable RRAMs , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[32]  Johannes Schemmel,et al.  Is a 4-Bit Synaptic Weight Resolution Enough? – Constraints on Enabling Spike-Timing Dependent Plasticity in Neuromorphic Hardware , 2012, Front. Neurosci..

[33]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[34]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[35]  Gert Cauwenberghs,et al.  Event-driven contrastive divergence for spiking neuromorphic systems , 2013, Front. Neurosci..

[36]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[37]  Timothée Masquelier,et al.  STDP-based spiking deep neural networks for object recognition , 2016, Neural Networks.

[38]  Jim D. Garside,et al.  SpiNNaker: A multi-core System-on-Chip for massively-parallel neural net simulation , 2012, Proceedings of the IEEE 2012 Custom Integrated Circuits Conference.

[39]  Somnath Paul,et al.  Event-driven random backpropagation: Enabling neuromorphic deep learning machines , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[40]  Maria Mitkova,et al.  Electrochemical Metallization Memories , 2016 .