Device-aware inference operations in SONOS nonvolatile memory arrays

Non-volatile memory arrays can deploy pre-trained neural network models for edge inference. However, these systems are affected by device-level noise and retention issues. Here, we examine damage caused by these effects, introduce a mitigation strategy, and demonstrate its use in fabricated array of SONOS (Silicon-Oxide-Nitride-Oxide-Silicon) devices. On MNIST, fashion-MNIST, and CIFAR-10 tasks, our approach increases resilience to synaptic noise and drift. We also show strong performance can be realized with ADCs of 5-8 bits precision.

[1]  Jason Cong,et al.  Scaling for edge inference of deep neural networks , 2018 .

[2]  Steven J. Plimpton,et al.  Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator , 2017, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[3]  Jie Lin,et al.  Noise Injection Adaption: End-to-End ReRAM Crossbar Non-ideal Effect Adaption for Neural Network Mapping , 2019, 2019 56th ACM/IEEE Design Automation Conference (DAC).

[4]  Vivienne Sze,et al.  Design Considerations for Efficient Deep Neural Networks on Processing-in-Memory Accelerators , 2019, 2019 IEEE International Electron Devices Meeting (IEDM).

[5]  Philippe Talatchian,et al.  Energy-efficient stochastic computing with superparamagnetic tunnel junctions , 2020, Physical review applied.

[6]  M. Janai,et al.  Data retention reliability model of NROM nonvolatile memory products , 2004, IEEE Transactions on Device and Materials Reliability.

[7]  Dmitri B. Strukov,et al.  Improving Noise Tolerance of Mixed-Signal Neural Networks , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[8]  Meng Chuan Lee,et al.  Charge Loss Mechanisms of Nitride-Based Charge Trap Flash Memory Devices , 2013, IEEE Transactions on Electron Devices.

[9]  Matthew J. Marinella,et al.  Using Floating-Gate Memory to Train Ideal Accuracy Neural Networks , 2019, IEEE Journal on Exploratory Solid-State Computational Devices and Circuits.

[10]  Farnood Merrikh-Bayat,et al.  High-Performance Mixed-Signal Neurocomputing With Nanoscale Floating-Gate Memory Cell Arrays , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[12]  Bohyung Han,et al.  Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization , 2017, NIPS.

[13]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[14]  Damien Querlioz,et al.  Digital Biologically Plausible Implementation of Binarized Neural Networks With Differential Hafnium Oxide Resistive Memory Arrays , 2019, Frontiers in Neuroscience.

[15]  Craig M. Vineyard,et al.  Training deep neural networks for binary communication with the Whetstone method , 2019 .

[16]  Steven J. Plimpton,et al.  Resistive memory device requirements for a neural algorithm accelerator , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[17]  Tayfun Gokmen,et al.  Training large-scale ANNs on simulated resistive crossbar arrays , 2019, ArXiv.

[18]  M. Janai,et al.  Relaxation of localized charge in trapping-based nonvolatile memory devices , 2008, 2008 IEEE International Reliability Physics Symposium.

[19]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[20]  Chih-Yuan Lu,et al.  Data retention behavior of a SONOS type two-bit storage flash memory cell , 2001, International Electron Devices Meeting. Technical Digest (Cat. No.01CH37224).