Resistive Memories for Spike-Based Neuromorphic Circuits

In the last decade machine learning algorithms have proven unprecedented performance to solve many real-world detection and classification tasks, for example in image or speech recognition. Despite these advances, there are still some deficits. First, these algorithms require significant memory access thus ruling out an implementation using standard platforms (e.g. GPUs, FPGAs) for embedded applications. Second, most machine leaning algorithms need to be trained with huge data sets (supervised learning). Resistive memories (RRAM) have demonstrated to be a promising candidate to overcome both these constrains. RRAM arrays can act as a dot product accelerator, which is one of the main building blocks in neuromorphic computing systems. This approach could provide improvements in power and speed with respect to the GPU-based networks. Moreover RRAM devices are promising candidates to emulate synaptic plasticity, the capability of synapses to enhance or diminish their connectivity between neurons, which is widely believed to be the basis for learning and memory in the brain. Neural systems exhibit various types and time periods of plasticity, e.g. synaptic modifications can last anywhere from seconds to days or months. In this work we proposed an architecture that implements both Short- and Long-Term Plasticity rules (STP and LTP) using RRAM arrays. We showed the benefits of utilizing both kinds of plasticity with two different applications, visual pattern extraction and decoding of neural signals. LTP allows the neural networks to learn patterns without training data set (unsupervised learning), and STP makes the learning process very robust against environmental noise.

[1]  Tobi Delbrück,et al.  A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor , 2008, IEEE Journal of Solid-State Circuits.

[2]  T. Delbruck,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 1 , 2022 .

[3]  T. Serrano-Gotarredona,et al.  STDP and STDP variations with memristors for spiking neuromorphic learning systems , 2013, Front. Neurosci..

[4]  E. Vianello,et al.  Spiking Neural Networks Based on OxRAM Synapses for Real-Time Unsupervised Spike Sorting , 2016, Frontiers in neuroscience.

[5]  Johannes Schemmel,et al.  A wafer-scale neuromorphic hardware system for large-scale neural modeling , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[6]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  E. Vianello,et al.  HfO2-Based OxRAM Devices as Synapses for Convolutional Neural Networks , 2015, IEEE Transactions on Electron Devices.

[9]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[10]  Nicholas D. Lane,et al.  An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices , 2015, IoT-App@SenSys.

[11]  Manan Suri,et al.  Exploiting Intrinsic Variability of Filamentary Resistive Memory for Extreme Learning Machine Architectures , 2015, IEEE Transactions on Nanotechnology.

[12]  G. Ghibaudo,et al.  Investigation of the potentialities of Vertical Resistive RAM (VRRAM) for neuromorphic applications , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[13]  Damien Querlioz,et al.  Bioinspired Programming of Memory Devices for Implementing an Inference Engine , 2015, Proceedings of the IEEE.

[14]  Karlheinz Meier,et al.  A mixed-signal universal neuromorphic computing system , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[15]  B. DeSalvo,et al.  Experimental demonstration of short and long term synaptic plasticity using OxRAM multi k-bit arrays for reliable detection in highly noisy input data , 2016, 2016 IEEE International Electron Devices Meeting (IEDM).

[16]  E. Vianello,et al.  Variability-tolerant Convolutional Neural Network for Pattern Recognition applications based on OxRAM synapses , 2014, 2014 IEEE International Electron Devices Meeting.

[17]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[18]  H. Markram,et al.  The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Elisa Vianello,et al.  On the impact of OxRAM-based synapses variability on convolutional neural networks performance , 2015, Proceedings of the 2015 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH´15).

[20]  Damien Querlioz,et al.  Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity , 2012, Neural Networks.

[21]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[22]  Damien Querlioz,et al.  Unsupervised features extraction from asynchronous silicon retina through Spike-Timing-Dependent Plasticity , 2011, The 2011 International Joint Conference on Neural Networks.

[23]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[24]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[25]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[26]  G. Cibrario,et al.  Fundamental variability limits of filament-based RRAM , 2016, 2016 IEEE International Electron Devices Meeting (IEDM).

[27]  Robert Legenstein,et al.  A compound memristive synapse model for statistical learning through STDP in spiking neural networks , 2014, Front. Neurosci..

[28]  E. Vianello,et al.  Bio-Inspired Stochastic Computing Using Binary CBRAM Synapses , 2013, IEEE Transactions on Electron Devices.

[29]  B. Giraud,et al.  Resistive Memories for Ultra-Low-Power embedded computing design , 2014, 2014 IEEE International Electron Devices Meeting.