Effect of Asymmetric Nonlinearity Dynamics in RRAMs on Spiking Neural Network Performance

Crossbar-based Resistive Random Access Memory (RRAM) array is a promising candidate for fast and efficient implementation of the vector-matrix multiplication, an essential step in a wide variety of workloads. However, several RRAM devices, demonstrating promising synaptic behaviors, are characterized by nonlinear and asymmetric update dynamics, which is a major obstacle for large-scale deployment in neural networks, especially for online learning tasks. In this work, we first introduce a memristive Spiking Neural Network (SNN) with local learning. Then, we study the effect of this asymmetric and nonlinear behavior on the spiking neural network performance and propose a method to overcome the performance degradation without extra nonlinearity cancellation hardware and read cycles. The performance of the proposed method approaches the baseline performance with 1 ∼ 2% drop in recognition accuracy.

[1]  Jiaming Zhang,et al.  Analogue signal and image processing with large memristor crossbars , 2017, Nature Electronics.

[2]  Colin J. Akerman,et al.  Random synaptic feedback weights support error backpropagation for deep learning , 2016, Nature Communications.

[3]  Gert Cauwenberghs,et al.  Reverse engineering the cognitive brain , 2013, Proceedings of the National Academy of Sciences.

[4]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[5]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[6]  Johannes Schemmel,et al.  Demonstrating Hybrid Learning in a Flexible Neuromorphic Hardware System , 2016, IEEE Transactions on Biomedical Circuits and Systems.

[7]  Qing Wu,et al.  Efficient and self-adaptive in-situ learning in multilayer memristor neural networks , 2018, Nature Communications.

[8]  Arild Nøkland,et al.  Training Neural Networks with Local Error Signals , 2019, ICML.

[9]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[10]  Somnath Paul,et al.  Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines , 2016, Front. Neurosci..

[11]  Pritish Narayanan,et al.  Deep Learning with Limited Numerical Precision , 2015, ICML.

[12]  Hyunsang Hwang,et al.  TiOx-Based RRAM Synapse With 64-Levels of Conductance and Symmetric Conductance Change by Adopting a Hybrid Pulse Scheme for Neuromorphic Computing , 2016, IEEE Electron Device Letters.

[13]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[14]  Shimeng Yu,et al.  Neuro-Inspired Computing With Emerging Nonvolatile Memorys , 2018, Proceedings of the IEEE.

[15]  E. Neftci,et al.  Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective , 2019, Memristive Devices for Brain-Inspired Computing.

[16]  Gert Cauwenberghs,et al.  Deep Supervised Learning Using Local Errors , 2017, Front. Neurosci..

[17]  Shimeng Yu,et al.  Resistive Memory-Based Analog Synapse: The Pursuit for Linear and Symmetric Weight Update , 2018, IEEE Nanotechnology Magazine.

[18]  Ahmed M. Eltawil,et al.  Independent Component Analysis Using RRAMs , 2019, IEEE Transactions on Nanotechnology.

[19]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[20]  Emre Neftci,et al.  Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-based optimization to spiking neural networks , 2019, IEEE Signal Processing Magazine.

[21]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[22]  J. Yang,et al.  Memristive crossbar arrays for brain-inspired computing , 2019, Nature Materials.

[23]  P. Pavan,et al.  A Novel Program-Verify Algorithm for Multi-Bit Operation in HfO2 RRAM , 2015, IEEE Electron Device Letters.