Adaptive Quantization as a Device-Algorithm Co-Design Approach to Improve the Performance of In-Memory Unsupervised Learning With SNNs

Off-chip memory access is the primary bottleneck toward accelerating neural network operations and reducing energy consumption. In-memory training and computation using emerging nonvolatile memories (eNVMs) have been proposed to address this problem. However, a small number of conductance states limit in-memory online learning performance. Here, we introduce a device-algorithm co-design approach and its application to phase change memory (PCM) for improving learning accuracy. We present an adaptive quantization method, which compensates the accuracy loss due to limited conductance levels and enables high-accuracy unsupervised learning with low-precision eNVM devices. We develop a spiking neural network framework for NeuroSim platform to compare online learning performance of PCM arrays for analog and digital implementations and benchmark the tradeoffs in energy consumption, latency, and area.

[1]  Max Welling,et al.  Relaxed Quantization for Discretized Neural Networks , 2018, ICLR.

[2]  Shimeng Yu,et al.  Mitigating effects of non-ideal synaptic device characteristics for on-chip learning , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[3]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[4]  Joel Emer,et al.  Eyeriss: an Energy-efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks Accessed Terms of Use , 2022 .

[5]  Eunhyeok Park,et al.  Weighted-Entropy-Based Quantization for Deep Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Bipin Rajendran,et al.  Spiking neural networks for handwritten digit recognition - Supervised learning and network optimization , 2018, Neural Networks.

[7]  Duygu Kuzum,et al.  Drift-Enhanced Unsupervised Learning of Handwritten Digits in Spiking Neural Network With PCM Synapses , 2018, IEEE Electron Device Letters.

[8]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[9]  Xing Lan,et al.  Zero-static power radio-frequency switches based on MoS2 atomristors , 2018, Nature Communications.

[10]  Xiaochen Peng,et al.  NeuroSim: A Circuit-Level Macro Model for Benchmarking Neuro-Inspired Architectures in Online Learning , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[11]  Zhongfeng Wang,et al.  Efficient Hardware Architectures for Deep Convolutional Neural Network , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[12]  H. Hwang,et al.  Improved Synaptic Behavior Under Identical Pulses Using AlOx/HfO2 Bilayer RRAM Array for Neuromorphic Systems , 2016, IEEE Electron Device Letters.

[13]  Dong-Ho Kang,et al.  Poly-4-vinylphenol (PVP) and Poly(melamine-co-formaldehyde) (PMF)-Based Atomic Switching Device and Its Application to Logic Gate Circuits with Low Operating Voltage. , 2017, ACS applied materials & interfaces.

[14]  Chung Lam,et al.  Experimental demonstration of array-level learning with phase change synaptic devices , 2013, 2013 IEEE International Electron Devices Meeting.

[15]  Myungsoo Kim,et al.  Atomristor: Nonvolatile Resistance Switching in Atomic Sheets of Transition Metal Dichalcogenides. , 2018, Nano letters.

[16]  N. Panwar,et al.  PCMO-Based RRAM and NPN Bipolar Selector as Synapse for Energy Efficient STDP , 2017, IEEE Electron Device Letters.

[17]  N Gong,et al.  Signal and noise extraction from analog memory elements for neuromorphic computing , 2018, Nature Communications.

[18]  Wei Lu,et al.  Neuromorphic Computing Using Memristor Crossbar Networks: A Focus on Bio-Inspired Approaches , 2018, IEEE Nanotechnology Magazine.

[19]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[20]  H.-S. Philip Wong,et al.  Energy efficient programming of nanoelectronic synaptic devices for large-scale implementation of associative and temporal sequence learning , 2011, 2011 International Electron Devices Meeting.

[21]  Gökmen Tayfun,et al.  Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices: Design Considerations , 2016, Front. Neurosci..