Multi-ReRAM Synapses for Artificial Neural Network Training

Metal-oxide-based resistive memory devices (ReRAM) are being actively researched as synaptic elements of neuromorphic co-processors for training deep neural networks (DNNs). However, device-level non-idealities are posing significant challenges. In this work we present a multi-ReRAM-based synaptic architecture with a counter-based arbitration scheme that shows significant promise. We present a 32×2 crossbar array comprising Pt/HfO2/Ti/TiN-based ReRAM devices with multi-level storage capability and bidirectional conductance response. We study the device characteristics in detail and model the conductance response. We show through simulations that an in-situ trained DNN with a multi-ReRAM synaptic architecture can perform handwritten digit classification task with high accuracies, only 2% lower than software simulations using floating point precision, despite the stochasticity, nonlinearity and large conductance change granularity associated with the devices. Moreover, we show that a network can achieve accuracies > 80% even with just binary ReRAM devices with this architecture.

[1]  Yusuf Leblebici,et al.  Neuromorphic computing with multi-memristive synapses , 2017, Nature Communications.

[2]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[3]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[4]  Yusuf Leblebici,et al.  Stochastic weight updates in phase-change memory-based synapses and their influence on artificial neural networks , 2017, 2017 13th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME).

[5]  Olivier Bichler,et al.  Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction , 2011, 2011 International Electron Devices Meeting.

[6]  Abu Sebastian,et al.  Tutorial: Brain-inspired computing using phase-change memory devices , 2018, Journal of Applied Physics.

[7]  R. Waser,et al.  Nanoionics-based resistive switching memories. , 2007, Nature materials.

[8]  B. Rajendran,et al.  Supervised learning in spiking neural networks with MLC PCM synapses , 2017, 2017 75th Annual Device Research Conference (DRC).

[9]  Richard George,et al.  Spike-driven threshold-based learning with memristive synapses and neuromorphic silicon neurons , 2018, Journal of Physics D: Applied Physics.

[10]  Yusuf Leblebici,et al.  Chip-level CMOS co-integration of ReRAM-based non-volatile memories , 2016, 2016 12th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME).

[11]  龙世兵.  Quantum-size effects in hafnium-oxide resistive switching , 2013 .

[12]  R. Cavin,et al.  Scaling limits of resistive memories , 2011, Nanotechnology.

[13]  Y. Leblebici,et al.  Large-scale neural networks implemented with non-volatile memory as the synaptic weight element: Comparative performance analysis (accuracy, speed, and power) , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[14]  Evangelos Eleftheriou,et al.  Mixed-precision architecture based on computational memory for training deep neural networks , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[15]  S. Ambrogio,et al.  Spike-timing dependent plasticity in a transistor-selected resistive switching memory , 2013, Nanotechnology.

[16]  Duane Mills,et al.  19.7 A 16Gb ReRAM with 200MB/s write and 1GB/s read in 27nm technology , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[17]  N Gong,et al.  Signal and noise extraction from analog memory elements for neuromorphic computing , 2018, Nature Communications.

[18]  Yusuf Leblebici,et al.  The key impact of incorporated Al2O3 barrier layer on W-based ReRAM switching performance , 2018, 2018 14th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME).

[19]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[20]  R. Waser,et al.  Resistive Switching: From Fundamentals of Nanoionic Redox Processes to Memristive Device Applications , 2016 .

[21]  Shimeng Yu,et al.  An Electronic Synapse Device Based on Metal Oxide Resistive Switching Memory for Neuromorphic Computation , 2011, IEEE Transactions on Electron Devices.

[22]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[23]  Jiaming Zhang,et al.  Analogue signal and image processing with large memristor crossbars , 2017, Nature Electronics.

[24]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[25]  R. Waser,et al.  Nanoscale cation motion in TaO(x), HfO(x) and TiO(x) memristive systems. , 2016, Nature nanotechnology.

[26]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.