Performance Analysis of Spiking RBM with Measurement-Based Phase Change Memory Model

Phase change memory (PCM) is arguably one of the most promising non-volatile memories which can be used in neuromorphic applications. When we use PCM as analog synaptic elements, non-ideality impact should be carefully taken into account. In this paper, we investigate the impact of such non-ideality items as nonlinear weight update and resistance drift on training accuracy in a spiking restricted Boltzmann machine (RBM). In addition to ideal PCM model, actual measurement-based PCM characteristics is used for this study. The resistance drift affects training accuracy, especially in a widely distributed read interval time caused by spatial and temporal sparse spike activities during training in spiking RBM. Our simulation results show that the training accuracy worsens with the increase of the weight-update nonlinearity or the resistance-drift coefficient. However, the results also suggest we can expect more than 8.95% improvement in training accuracy. This potential improvement will be possible if we use linearity-improved confined PCM cells whose median resistance drift coefficient is 0.005, compared to the resistance drift coefficient of more than 0.02 in typical existing PCM cells.

[1]  D. Querlioz,et al.  Visual Pattern Extraction Using Energy-Efficient “2-PCM Synapse” Neuromorphic Architecture , 2012, IEEE Transactions on Electron Devices.

[2]  Damien Querlioz,et al.  Impact of PCM resistance-drift in neuromorphic systems and drift-mitigation strategy , 2013, 2013 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[3]  R. Jordan,et al.  NVM neuromorphic core with 64k-cell (256-by-256) phase change memory synaptic array with on-chip neuron circuits for continuous in-situ learning , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[4]  Yusuf Leblebici,et al.  Neuromorphic computing with multi-memristive synapses , 2017, Nature Communications.

[5]  Gert Cauwenberghs,et al.  Event-driven contrastive divergence for spiking neuromorphic systems , 2013, Front. Neurosci..

[6]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[7]  Megumi Ito,et al.  NVM Weight Variation Impact on Analog Spiking Neural Network Chip , 2018, ICONIP.

[8]  Chih-Yuan Lu,et al.  Performance Impacts of Analog ReRAM Non-ideality on Neuromorphic Computing , 2019, IEEE Transactions on Electron Devices.

[9]  Ryutaro Yasuhara,et al.  A 4M Synapses integrated Analog ReRAM based 66.5 TOPS/W Neural-Network Processor with Cell Current Controlled Writing and Flexible Network Architecture , 2018, 2018 IEEE Symposium on VLSI Technology.

[10]  Olivier Bichler,et al.  Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction , 2011, 2011 International Electron Devices Meeting.

[11]  Abu Sebastian,et al.  Tutorial: Brain-inspired computing using phase-change memory devices , 2018, Journal of Applied Physics.

[12]  E. Eleftheriou,et al.  A phase-change memory model for neuromorphic computing , 2018, Journal of Applied Physics.

[13]  Megumi Ito,et al.  Lightweight Refresh Method for PCM-based Neuromorphic Circuits , 2018, 2018 IEEE 18th International Conference on Nanotechnology (IEEE-NANO).

[14]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[15]  Yusuf Leblebici,et al.  Stochastic weight updates in phase-change memory-based synapses and their influence on artificial neural networks , 2017, 2017 13th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME).

[16]  S. Ambrogio,et al.  Confined PCM-based Analog Synaptic Devices offering Low Resistance-drift and 1000 Programmable States for Deep Learning , 2019, 2019 Symposium on VLSI Technology.

[17]  Kun Yue,et al.  A brain-plausible neuromorphic on-the-fly learning system implemented with magnetic domain wall analog memristors , 2019, Science Advances.

[18]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[19]  P. Narayanan,et al.  Recent progress in analog memory-based accelerators for deep learning , 2018, Journal of Physics D: Applied Physics.

[20]  Damien Querlioz,et al.  Using Memristors for Robust Local Learning of Hardware Restricted Boltzmann Machines , 2019, Scientific Reports.

[21]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.