Probabilistic Deep Spiking Neural Systems Enabled by Magnetic Tunnel Junction

Deep spiking neural networks are becoming increasingly powerful tools for cognitive computing platforms. However, most of the existing studies on such computing models are developed with limited insights on the underlying hardware implementation, resulting in area and power expensive designs. Although several neuromimetic devices emulating neural operations have been proposed recently, their functionality has been limited to very simple neural models that may prove to be inefficient at complex recognition tasks. In this paper, we venture into the relatively unexplored area of utilizing the inherent device stochasticity of such neuromimetic devices to model complex neural functionalities in a probabilistic framework in the time domain. We consider the implementation of a deep spiking neural network capable of performing high-accuracy and lowlatency classification tasks, where the neural computing unit is enabled by the stochastic switching behavior of a magnetic tunnel junction. The simulation studies indicate an energy improvement of 20× over a baseline CMOS design in 45-nm technology.

[1]  W. Marsden I and J , 2012 .

[2]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[3]  D. Ralph,et al.  Current-induced switching of perpendicularly magnetized magnetic layers using spin torque from the spin Hall effect. , 2012, Physical review letters.

[4]  D. Ralph,et al.  Spin transfer torque devices utilizing the giant spin Hall effect of tungsten , 2012, 1208.1711.

[5]  Xuanyao Fong,et al.  KNACK: A hybrid spin-charge mixed-mode simulator for evaluating different genres of spin-transfer torque MRAM bit-cells , 2011, 2011 International Conference on Simulation of Semiconductor Processes and Devices.

[6]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[7]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[8]  Slonczewski Conductance and exchange coupling of two ferromagnets separated by a tunneling barrier. , 1989, Physical review. B, Condensed matter.

[9]  Kaushik Roy,et al.  STT-SNN: A Spin-Transfer-Torque Based Soft-Limiting Non-Linear Neuron for Low-Power Artificial Neural Networks , 2014, IEEE Transactions on Nanotechnology.

[10]  Kaushik Roy,et al.  Spin Neurons: A Possible Path to Energy-Efficient Neuromorphic Computers , 2013, ArXiv.

[11]  K. Roy,et al.  Spin-Based Neuron Model With Domain-Wall Magnets as Synapse , 2012, IEEE Transactions on Nanotechnology.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  J. Hirsch Spin Hall Effect , 1999, cond-mat/9906160.

[14]  Rasmus Berg Palm,et al.  Prediction as a candidate for learning deep hierarchical models of data , 2012 .

[15]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[16]  D. Ralph,et al.  Spin-Torque Switching with the Giant Spin Hall Effect of Tantalum , 2012, Science.

[17]  T. Hasegawa,et al.  Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. , 2011, Nature materials.

[18]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[19]  Kaushik Roy,et al.  Spin Orbit Torque Based Electronic Neuron , 2014, ArXiv.

[20]  Kaushik Roy,et al.  Spin-Orbit Torque Induced Spike-Timing Dependent Plasticity , 2014, ArXiv.

[21]  Wei Lu,et al.  Short-term Memory to Long-term Memory Transition in a Nanoscale Memristor , 2022 .

[22]  Werner Scholz,et al.  Micromagnetic simulation of thermally activated switching in fine particles , 2001 .

[23]  Geoffrey W. Burr,et al.  Nanoscale electronic synapses using phase change devices , 2013, JETC.

[24]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[25]  Hideo Ohno,et al.  Current-induced effective field in perpendicularly magnetized Ta/CoFeB/MgO wire , 2011 .

[26]  Chris Eliasmith,et al.  Spiking Deep Networks with LIF Neurons , 2015, ArXiv.

[27]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[28]  Kaushik Roy,et al.  Short-Term Plasticity and Long-Term Potentiation in Magnetic Tunnel Junctions: Towards Volatile Synapses , 2015, ArXiv.