Linear Optimization for Memristive Device in Neuromorphic Hardware

Memristors offer advantages as a hardware solution for neuromorphic computing, however, their nonlinear property makes the weight update difficult and reduces the accuracy of a neural network. A piecewise linear (PL) method is proposed in this paper to mitigate the nonlinear effect of memristors by calculating the weight update parameters along a piecewise line, which reduces errors in the weight update process. It is a simple but efficient method for the nonlinearity mitigation without reading the current conductance of the memristor in each updating, thereby avoiding complex peripheral circuits. The PL methods with respectively with 2-segment, 3-segment, and 4-segment models in two split points selection strategies are investigated, and the results show that under different nonlinearity, the PL method improves the recognition accuracy of MNIST handwriting digits to 87.87%-95.05%, as compared to 10.77%-73.18% of the cases without PL method. Finally, it concludes that the more segments in PL methods, the less weight deviation caused by the non-linearity of the synapse device.

[1]  Yusuf Leblebici,et al.  Large-scale neural networks implemented with Non-Volatile Memory as the synaptic weight element: Impact of conductance response , 2016, 2016 46th European Solid-State Device Research Conference (ESSDERC).

[2]  Byung-Gook Park,et al.  High-Density and Near-Linear Synaptic Device Based on a Reconfigurable Gated Schottky Diode , 2017, IEEE Electron Device Letters.

[3]  G. W. Burr,et al.  Experimental demonstration and tolerancing of a large-scale neural network (165,000 synapses), using phase-change memory as the synaptic weight element , 2015, 2014 IEEE International Electron Devices Meeting.

[4]  Ru Huang,et al.  Engineering incremental resistive switching in TaOx based memristors for brain-inspired computing. , 2016, Nanoscale.

[5]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[6]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[7]  M. B. Ghaznavi-Ghoushchi,et al.  A high-precision time-domain RRAM state control approach , 2018, Microelectron. J..

[8]  L. Chua.  Memristor-The missing circuit element , 1971 .

[9]  I-Ting Wang,et al.  3D Ta/TaOx/TiO2/Ti synaptic array and linearity tuning of weight update for hardware neural network applications , 2016, Nanotechnology.

[10]  H. Hwang,et al.  Improved Synaptic Behavior Under Identical Pulses Using AlOx/HfO2 Bilayer RRAM Array for Neuromorphic Systems , 2016, IEEE Electron Device Letters.

[11]  Hyongsuk Kim,et al.  Memristor-based multilevel memory , 2010, 2010 12th International Workshop on Cellular Nanoscale Networks and their Applications (CNNA 2010).

[12]  Shimeng Yu,et al.  Mitigating effects of non-ideal synaptic device characteristics for on-chip learning , 2015, 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[13]  Qing Wu,et al.  Efficient and self-adaptive in-situ learning in multilayer memristor neural networks , 2018, Nature Communications.

[14]  J. Kim,et al.  Neuromorphic speech systems using advanced ReRAM-based synapse , 2013, 2013 IEEE International Electron Devices Meeting.

[15]  Catherine E. Graves,et al.  Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine , 2018, Advanced materials.

[16]  Shimeng Yu,et al.  Improving Analog Switching in HfOx-Based Resistive Memory With a Thermal Enhanced Layer , 2017, IEEE Electron Device Letters.

[17]  Jiaming Zhang,et al.  Analogue signal and image processing with large memristor crossbars , 2017, Nature Electronics.

[18]  Xiaochen Peng,et al.  NeuroSim: A Circuit-Level Macro Model for Benchmarking Neuro-Inspired Architectures in Online Learning , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[19]  Fundamental Limits to Moore's Law , 2015, 1511.05956.

[20]  Shimeng Yu,et al.  Fully parallel write/read in resistive synaptic array for accelerating on-chip learning , 2015, Nanotechnology.

[21]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[22]  Xiaochen Peng,et al.  NeuroSim+: An integrated device-to-algorithm framework for benchmarking synaptic devices and array architectures , 2017, 2017 IEEE International Electron Devices Meeting (IEDM).