Semi-supervised learning and inference in domain-wall magnetic tunnel junction (DW-MTJ) neural networks

Advances in machine intelligence have sparked interest in hardware accelerators to implement these algorithms, yet embedded electronics have stringent power, area budgets, and speed requirements that may limit non- volatile memory (NVM) integration. In this context, the development of fast nanomagnetic neural networks using minimal training data is attractive. Here, we extend an inference-only proposal using the intrinsic physics of domain-wall MTJ (DW-MTJ) neurons for online learning to implement fully unsupervised pattern recognition operation, using winner-take-all networks that contain either random or plastic synapses (weights). Meanwhile, a read-out layer trains in a supervised fashion. We find our proposed design can approach state-of-the-art success on the task relative to competing memristive neural network proposals, while eliminating much of the area and energy overhead that would typically be required to build the neuronal layers with CMOS devices.

[1]  David Kappel,et al.  STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning , 2014, PLoS Comput. Biol..

[2]  Christopher H. Bennett,et al.  Graded-Anisotropy-Induced Magnetic Domain Wall Drift for an Artificial Spintronic Leaky Integrate-and-Fire Neuron , 2019, IEEE Journal on Exploratory Solid-State Computational Devices and Circuits.

[3]  Damien Querlioz,et al.  Contrasting Advantages of Learning With Random Weights and Backpropagation in Non-Volatile Memory Neural Networks , 2019, IEEE Access.

[4]  Jacques-Olivier Klein,et al.  Bioinspired networks with nanoscale memristive devices that combine the unsupervised and supervised learning approaches , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[5]  Steven J. Plimpton,et al.  Resistive memory device requirements for a neural algorithm accelerator , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[6]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[7]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[8]  Kevin Garello,et al.  Spin-orbit torque magnetization switching of a three-terminal perpendicular magnetic tunnel junction , 2013, 1310.8235.

[9]  Ankur Srivastava,et al.  In Situ Stochastic Training of MTJ Crossbars With Machine Learning Algorithms , 2019, ACM J. Emerg. Technol. Comput. Syst..

[10]  Ilya Krivorotov,et al.  Switching current reduction using perpendicular anisotropy in CoFeB-MgO magnetic tunnel junctions , 2011 .

[11]  Damien Querlioz,et al.  Bioinspired Programming of Memory Devices for Implementing an Inference Engine , 2015, Proceedings of the IEEE.

[12]  Themis Prodromakis,et al.  Analog Memristive Synapse in Spiking Networks Implementing Unsupervised Learning , 2016, Front. Neurosci..

[13]  Steve Furber,et al.  Large-scale neuromorphic computing systems , 2016, Journal of neural engineering.

[14]  Ali Khiat,et al.  Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses , 2016, Nature Communications.

[15]  Mark Horowitz,et al.  1.1 Computing's energy problem (and what we can do about it) , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[16]  Luca Maria Gambardella,et al.  Convolutional Neural Network Committees for Handwritten Character Classification , 2011, 2011 International Conference on Document Analysis and Recognition.

[17]  Joseph S. Friedman,et al.  Magnetic domain wall neuron with lateral inhibition , 2018, Journal of Applied Physics.

[18]  M. Prezioso,et al.  Modeling and implementation of firing-rate neuromorphic-network classifiers with bilayer Pt/Al2O3/TiO2−x/Pt Memristors , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[19]  Kaushik Roy,et al.  Proposal for an All-Spin Artificial Neural Network: Emulating Neural and Synaptic Functionalities Through Domain Wall Motion in Ferromagnets , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[20]  C. A. Ross,et al.  Logic circuit prototypes for three-terminal magnetic tunnel junctions with mobile domain walls , 2016, Nature Communications.

[21]  D. Querlioz,et al.  Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices , 2013, IEEE Transactions on Nanotechnology.

[22]  Pritish Narayanan,et al.  Experimental Demonstration and Tolerancing of a Large-Scale Neural Network (165 000 Synapses) Using Phase-Change Memory as the Synaptic Weight Element , 2014, IEEE Transactions on Electron Devices.

[23]  Jacques-Olivier Klein,et al.  Spin-Transfer Torque Magnetic Memory as a Stochastic Memristive Synapse for Neuromorphic Systems , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[24]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[25]  Gert Cauwenberghs,et al.  Memristors Empower Spiking Neurons With Stochasticity , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[26]  Kaushik Roy,et al.  Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning , 2016, Scientific Reports.

[27]  Jacques-Olivier Klein,et al.  Physical Realization of a Supervised Learning System Built with Organic Memristive Synapses , 2016, Scientific Reports.

[28]  C. Ross,et al.  Low Energy Magnetic Domain Wall Logic in Short, Narrow, Ferromagnetic Wires , 2012, IEEE Magnetics Letters.