Mask Technique for Fast and Efficient Training of Binary Resistive Crossbar Arrays

Resistive crossbar arrays, despite their advantages such as parallel processing, low power, and high density, suffer from the sneak path problem created by IR voltage drops. The crossbar non-idealities considerably degrade the accuracy of matrix multiplication operation, which is the cornerstone of hardware accelerated neural networks. In this paper, we propose a novel technique to capture the sneak path problem to enable off-chip learning and weight transfer. The proposed technique can be easily integrated into any neural network training framework. Performance results show a significant improvement after retraining the network with the proposed mask technique. Two mask solutions have been proposed and studied to capture the sneak path problem in resistive crossbar arrays. Both mask solutions were successfully able to achieve classification accuracy close to the baseline accuracy. The tradeoffs between the two solutions are discussed and compared in terms of accuracy, power, and area.

[1]  Xiaochen Peng,et al.  Fully parallel RRAM synaptic array for implementing binary neural network with (+1, −1) weights and (+1, 0) neurons , 2018, 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC).

[2]  Jean-Philippe Noël,et al.  SneakPath compensation circuit for programming and read operations in RRAM-based CrossPoint architectures , 2015, 2015 15th Non-Volatile Memory Technology Symposium (NVMTS).

[3]  Rajiv V. Joshi,et al.  An Energy-Efficient Digital ReRAM-Crossbar-Based CNN With Bitwise Parallelism , 2017, IEEE Journal on Exploratory Solid-State Computational Devices and Circuits.

[4]  Kaushik Roy,et al.  Rx-Caffe: Framework for evaluating and training Deep Neural Networks on Resistive Crossbars , 2018, ArXiv.

[5]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[6]  Daniel Josell,et al.  Size-Dependent Resistivity in Nanoscale Interconnects , 2009 .

[7]  Qing Wu,et al.  Efficient and self-adaptive in-situ learning in multilayer memristor neural networks , 2018, Nature Communications.

[8]  Xiaochen Peng,et al.  XNOR-RRAM: A scalable and parallel resistive synaptic architecture for binary neural networks , 2018, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[9]  Shimeng Yu,et al.  Neuro-Inspired Computing With Emerging Nonvolatile Memorys , 2018, Proceedings of the IEEE.

[10]  Ahmed M. Eltawil,et al.  Independent Component Analysis Using RRAMs , 2019, IEEE Transactions on Nanotechnology.

[11]  Nagarajan Raghavan,et al.  Recommended Methods to Study Resistive Switching Devices , 2018, Advanced Electronic Materials.

[12]  Ran El-Yaniv,et al.  Binarized Neural Networks , 2016, ArXiv.

[13]  Catherine E. Graves,et al.  Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine , 2018, Advanced materials.

[14]  Indranil Chakraborty,et al.  Technology Aware Training in Memristive Neuromorphic Systems for Nonideal Synaptic Crossbars , 2017, IEEE Transactions on Emerging Topics in Computational Intelligence.

[15]  Emmanuelle M. Grafals,et al.  Voltage divider effect for the improvement of variability and endurance of TaOx memristor , 2016, Scientific Reports.

[16]  Jongeun Lee,et al.  Cost-effective stochastic MAC circuits for deep neural networks , 2019, Neural Networks.

[17]  Ahmed M. Eltawil,et al.  Modeling and Analysis of Passive Switching Crossbar Arrays , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[18]  Pierre-Emmanuel Gaillardon,et al.  A Robust Digital RRAM-Based Convolutional Block for Low-Power Image Processing and Learning Applications , 2019, IEEE Transactions on Circuits and Systems I: Regular Papers.

[19]  Zhiwei Li,et al.  Binary neural network with 16 Mb RRAM macro chip for classification and online training , 2016, 2016 IEEE International Electron Devices Meeting (IEDM).

[20]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[21]  Dmitri B. Strukov,et al.  Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits , 2017, Nature Communications.

[22]  Ahmed M. Eltawil,et al.  Overcoming Crossbar Nonidealities in Binary Neural Networks Through Learning , 2018, 2018 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[23]  Baker Mohammad,et al.  Memristor Technology: Synthesis and Modeling for Sensing and Security Applications , 2017 .

[24]  Yu Wang,et al.  Binary convolutional neural network on RRAM , 2017, 2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC).

[25]  Mohammed A. Zidan,et al.  Parasitic Effect Analysis in Memristor-Array-Based Neuromorphic Systems , 2018, IEEE Transactions on Nanotechnology.

[26]  Chih-Cheng Chang,et al.  Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[27]  裕幸 飯田,et al.  International Technology Roadmap for Semiconductors 2003の要求清浄度について - シリコンウエハ表面と雰囲気環境に要求される清浄度, 分析方法の現状について - , 2004 .

[28]  Marian Verhelst,et al.  An Always-On 3.8 $\mu$ J/86% CIFAR-10 Mixed-Signal Binary CNN Processor With All Memory on Chip in 28-nm CMOS , 2019, IEEE Journal of Solid-State Circuits.