A quantization-aware regularized learning method in multilevel memristor-based neuromorphic computing system

In this work, we propose a regularized learning method that is able to take into account the deviation of the memristor-mapped synaptic weights from the target values determined during the training process. Experimental results obtained when utilizing the MNIST data set show that compared to the conventional learning method which considers the learning and mapping processes separately, our learning method can substantially improve the computation accuracy of the mapped two-layer multilayer perceptron (and LeNet-5) on multi-level memristor crossbars by 4.30% (11.05%) for binary representation, and by 0.40% (8.06%) for three-level representation.

[1]  L. Chua.  Memristor-The missing circuit element , 1971 .

[2]  Vibhore Vardhan,et al.  Power Consumption Breakdown on a Modern Laptop , 2004, PACS.

[3]  W. Jang,et al.  Low-Power and Highly Reliable Multilevel Operation in $ \hbox{ZrO}_{2}$ 1T1R RRAM , 2011, IEEE Electron Device Letters.

[4]  Wei Zhang,et al.  Digital-assisted noise-eliminating training for memristor crossbar-based analog neuromorphic computing engine , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).

[5]  Jiale Liang,et al.  Cross-Point Memory Array Without Cell Selectors—Device Characteristics and Data Storage Pattern Dependencies , 2010, IEEE Transactions on Electron Devices.

[6]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[7]  U-In Chung,et al.  Multi-level switching of triple-layered TaOx RRAM with excellent reliability for storage class memory , 2012, 2012 Symposium on VLSI Technology (VLSIT).

[8]  Hao Jiang,et al.  A spiking neuromorphic design with resistive crossbar , 2015, 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[9]  Yiran Chen,et al.  Vortex: Variation-aware training for memristor X-bar , 2015, 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[10]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  Yiran Chen,et al.  BSB training scheme implementation on memristor-based circuit , 2013, 2013 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA).

[13]  Sally A. McKee,et al.  Reflections on the memory wall , 2004, CF '04.

[14]  J. Yang,et al.  A compact modeling of TiO2-TiO2–x memristor , 2013 .

[15]  Kaushik Roy,et al.  Ultra low power associative computing with spin neurons and resistive crossbar memory , 2013, 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC).