A Device Non-Ideality Resilient Approach for Mapping Neural Networks to Crossbar Arrays

We propose a technology-independent method, referred to as adjacent connection matrix (ACM), to efficiently map signed weight matrices to non-negative crossbar arrays. When compared to same-hardware-overhead mapping methods, using ACM leads to improvements of up to 20% in training accuracy for ResNet-20 with the CIFAR-10 dataset when training with 5-bit precision crossbar arrays or lower. When compared with strategies that use two elements to represent a weight, ACM achieves comparable training accuracies, while also offering area and read energy reductions of 2.3× and 7×, respectively. ACM also has a mild regularization effect that improves inference accuracy in crossbar arrays without any retraining or costly device/variation-aware training.

[1]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[2]  Xiaochen Peng,et al.  NeuroSim+: An integrated device-to-algorithm framework for benchmarking synaptic devices and array architectures , 2017, 2017 IEEE International Electron Devices Meeting (IEDM).

[3]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[4]  Song Han,et al.  EIE: Efficient Inference Engine on Compressed Deep Neural Network , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[5]  Tao Zhang,et al.  PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[6]  Tayfun Gokmen,et al.  The Next Generation of Deep Learning Hardware: Analog Computing , 2019, Proceedings of the IEEE.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Shimeng Yu,et al.  Demonstration of Generative Adversarial Network by Intrinsic Random Noises of Analog RRAM Devices , 2018, 2018 IEEE International Electron Devices Meeting (IEDM).

[9]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[10]  Shimeng Yu,et al.  Resistive Memory-Based Analog Synapse: The Pursuit for Linear and Symmetric Weight Update , 2018, IEEE Nanotechnology Magazine.

[11]  Shimeng Yu,et al.  A ferroelectric field effect transistor based synaptic weight cell , 2018, Journal of Physics D: Applied Physics.

[12]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[13]  Chih-Cheng Chang,et al.  Mitigating Asymmetric Nonlinear Weight Update Effects in Hardware Neural Network Based on Analog Resistive Synapse , 2018, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[14]  Ahmed M. Eltawil,et al.  Independent Component Analysis Using RRAMs , 2019, IEEE Transactions on Nanotechnology.

[15]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[16]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[17]  Shuchang Zhou,et al.  DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients , 2016, ArXiv.

[18]  Gökmen Tayfun,et al.  Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices: Design Considerations , 2016, Front. Neurosci..