Robust RRAM-based In-Memory Computing in Light of Model Stability

Resistive random-access memory (RRAM)-based in-memory computing (IMC) architectures offer an energy-efficient solution for DNN acceleration. However, the performance of RRAM-based IMC is limited by device nonidealities, ADC precision, and algorithm properties. To address this, in this work, first, we perform statistical characterization of RRAM device variation and temporal degradation from 300mm wafers of a fully integrated CMOS/RRAM 1T1R test chip at 65nm. Through this, we build a realistic foundation to assess the robustness. Second, we develop a cross-layer simulation tool that incorporates device, circuit, architecture, and algorithm properties under a single roof for system evaluation. Finally, we propose a novel loss landscape-based DNN model selection for stability, which effectively tolerates device variations and achieves a post-mapping accuracy higher than that with 50% lower RRAM variations. We demonstrate the proposed method for different DNNs on both CIFAR-10 and CIFAR-100 datasets.

[1]  Kaushik Roy,et al.  GENIEx: A Generalized Approach to Emulating Non-Ideality in Memristive Xbars using Neural Networks , 2020, 2020 57th ACM/IEEE Design Automation Conference (DAC).

[2]  Swagath Venkataramani,et al.  PACT: Parameterized Clipping Activation for Quantized Neural Networks , 2018, ArXiv.

[3]  Frederick T. Chen,et al.  RRAM Defect Modeling and Failure Analysis Based on March Test and a Novel Squeeze-Search Scheme , 2015, IEEE Transactions on Computers.

[4]  Yu Cao,et al.  Interconnect-Aware Area and Energy Optimization for In-Memory Acceleration of DNNs , 2020, IEEE Design & Test.

[5]  Rajiv V. Joshi,et al.  Accurate Inference with Inaccurate RRAM Devices: Statistical Data, Model Transfer, and On-line Adaptation , 2020, 2020 57th ACM/IEEE Design Automation Conference (DAC).

[6]  Chih-Cheng Chang,et al.  Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[7]  Umit Y. Ogras,et al.  A Latency-Optimized Reconfigurable NoC for In-Memory Acceleration of DNNs , 2020, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[8]  Xiaoyu Sun,et al.  Impact of Non-Ideal Characteristics of Resistive Synaptic Devices on Implementing Convolutional Neural Networks , 2019, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[9]  Yu Cao,et al.  Noise-based Selection of Robust Inherited Model for Accurate Continual Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Shimeng Yu,et al.  Mitigating effects of non-ideal synaptic device characteristics for on-chip learning , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[11]  Hao Li,et al.  Visualizing the Loss Landscape of Neural Nets , 2017, NeurIPS.

[12]  Yandong Luo,et al.  Impact of Read Disturb on Multilevel RRAM based Inference Engine: Experiments and Model Prediction , 2020, 2020 IEEE International Reliability Physics Symposium (IRPS).

[13]  Nan Jiang,et al.  A detailed and flexible cycle-accurate Network-on-Chip simulator , 2013, 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[14]  Xiaochen Peng,et al.  DNN+NeuroSim: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators with Versatile Device Technologies , 2019, 2019 IEEE International Electron Devices Meeting (IEDM).

[15]  Karsten Beckmann,et al.  Improving the Memory Window/Resistance Variability Trade-Off for 65nm CMOS Integrated HfO2 Based Nanoscale RRAM Devices , 2019, 2019 IEEE International Integrated Reliability Workshop (IIRW).