Spin Orbit Torque Device based Stochastic Multi-bit Synapses for On-chip STDP Learning

As a large number of neurons and synapses are needed in spike neural network (SNN) design, emerging devices have been employed to implement synapses and neurons. In this paper, we present a stochastic multi-bit spin orbit torque (SOT) memory based synapse, where only one SOT device is switched for potentiation and depression using modified Gray code. The modified Gray code based approach needs only N devices to represent 2N levels of synapse weights. Early read termination scheme is also adopted to reduce the power consumption of training process by turning off less associated neurons and its ADCs. For MNIST dataset, with comparable classification accuracy, the proposed SNN architecture using 3-bit synapse achieves 68.7% reduction of ADC overhead compared to the conventional 8-level synapse.

[1]  Kaushik Roy,et al.  Hybrid Spintronic-CMOS Spiking Neural Network With On-Chip Learning: Devices, Circuits and Systems , 2015, ArXiv.

[2]  Yong Liu,et al.  A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[3]  B. Rajendran,et al.  Neuromorphic Computing Based on Emerging Memory Technologies , 2016, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[4]  Chris H. Kim,et al.  A technology-agnostic MTJ SPICE model with user-defined dimensions for STT-MRAM scalability studies , 2015, 2015 IEEE Custom Integrated Circuits Conference (CICC).

[5]  Kaushik Roy,et al.  Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning , 2016, Scientific Reports.

[6]  Youguang Zhang,et al.  Stochastic spintronic device based synapses and spiking neurons for neuromorphic computation , 2016, 2016 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  Robert Legenstein,et al.  A compound memristive synapse model for statistical learning through STDP in spiking neural networks , 2014, Front. Neurosci..

[9]  D. Ralph,et al.  Spin-Torque Switching with the Giant Spin Hall Effect of Tantalum , 2012, Science.

[10]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[11]  T. Branco,et al.  The probability of neurotransmitter release: variability and feedback control at single synapses , 2009, Nature Reviews Neuroscience.

[12]  Ki-Seung Lee,et al.  Thermally activated switching of perpendicular magnet by spin-orbit spin torque , 2014 .

[13]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[14]  Yongtae Kim,et al.  Neuromorphic Processors with Memristive Synapses , 2016, ACM J. Emerg. Technol. Comput. Syst..

[15]  Qian Wang,et al.  Energy efficient parallel neuromorphic architectures with approximate arithmetic on FPGA , 2017, Neurocomputing.

[16]  Jacques-Olivier Klein,et al.  Spin-Transfer Torque Magnetic Memory as a Stochastic Memristive Synapse for Neuromorphic Systems , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[17]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[18]  Anthony N. Burkitt,et al.  A Review of the Integrate-and-fire Neuron Model: I. Homogeneous Synaptic Input , 2006, Biological Cybernetics.

[19]  Zhaohui Huang,et al.  An architectural power estimator for analog-to-digital converters , 2004, IEEE International Conference on Computer Design: VLSI in Computers and Processors, 2004. ICCD 2004. Proceedings..