Encoding Neural and Synaptic Functionalities in Electron Spin: A Pathway to Efficient Neuromorphic Computing

Present day computers expend orders of magnitude more computational resources to perform various cognitive and perception related tasks that humans routinely perform everyday. This has recently resulted in a seismic shift in the field of computation where research efforts are being directed to develop a neurocomputer that attempts to mimic the human brain by nanoelectronic components and thereby harness its efficiency in recognition problems. Bridging the gap between neuroscience and nanoelectronics, this paper attempts to provide a review of the recent developments in the field of spintronic device based neuromorphic computing. Description of various spin-transfer torque mechanisms that can be potentially utilized for realizing device structures mimicking neural and synaptic functionalities is provided. A cross-layer perspective extending from the device to the circuit and system level is presented to envision the design of an All-Spin neuromorphic processor enabled with on-chip learning functionalities. Device-circuit-algorithm co-simulation framework calibrated to experimental results suggest that such All-Spin neuromorphic systems can potentially achieve almost two orders of magnitude energy improvement in comparison to state-of-the-art CMOS implementations.

[1]  K L Magleby,et al.  The effect of repetitive stimulation on facilitation of transmitter release at the frog neuromuscular junction , 1973, The Journal of physiology.

[2]  S. Watts,et al.  Latest Advances and Roadmap for In-Plane and Perpendicular STT-RAM , 2011, 2011 3rd IEEE International Memory Workshop (IMW).

[3]  Wolfgang Porod,et al.  Physical Implementation of Coherently Coupled Oscillator Networks , 2015, IEEE Journal on Exploratory Solid-State Computational Devices and Circuits.

[4]  H. Ohno,et al.  Tunnel magnetoresistance of 604% at 300K by suppression of Ta diffusion in CoFeB∕MgO∕CoFeB pseudo-spin-valves annealed at high temperature , 2008 .

[5]  D. Ralph,et al.  Spin transfer torque devices utilizing the giant spin Hall effect of tungsten , 2012, 1208.1711.

[6]  W. Butler,et al.  Large magnetoresistance in bcc Co/MgO/Co and FeCo/MgO/FeCo tunnel junctions , 2004, cond-mat/0409155.

[7]  M. Donahue,et al.  Head To Head Domain Wall Structures In Thin Magnetic Stripes , 1997, 1997 IEEE International Magnetics Conference (INTERMAG'97).

[8]  Yi Ji.  Spin injection, diffusion and detection in lateral spin valves , 2004 .

[9]  Kang L. Wang,et al.  Switching of perpendicular magnetization by spin-orbit torques in the absence of external magnetic fields. , 2013, Nature nanotechnology.

[10]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[11]  V. Cros,et al.  Spin-Torque Diode Measurements of MgO-Based Magnetic Tunnel Junctions with Asymmetric Electrodes , 2011, 1103.3207.

[12]  L. M. Ward,et al.  Stochastic resonance and sensory information processing: a tutorial and review of application , 2004, Clinical Neurophysiology.

[13]  Igor Zutic,et al.  Roadmap for Emerging Materials for Spintronic Device Applications , 2015, IEEE Transactions on Magnetics.

[14]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[15]  Kaushik Roy,et al.  Stochastic Spiking Neural Networks Enabled by Magnetic Tunnel Junctions: From Nontelegraphic to Telegraphic Switching Regimes , 2017 .

[16]  Yoshihiko Horio,et al.  Analogue spin–orbit torque device for artificial-neural-network-based associative memory operation , 2016 .

[17]  Kang L. Wang,et al.  VOLTAGE-CONTROLLED MAGNETIC ANISOTROPY IN SPINTRONIC DEVICES , 2012 .

[18]  Deepak Khosla,et al.  Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition , 2014, International Journal of Computer Vision.

[19]  Chiara Bartolozzi,et al.  Neuromorphic Electronic Circuits for Building Autonomous Cognitive Systems , 2014, Proceedings of the IEEE.

[20]  Kang L. Wang,et al.  Magnetization switching through giant spin-orbit torque in a magnetically doped topological insulator heterostructure. , 2014, Nature materials.

[21]  Sebastiaan van Dijken,et al.  Reversible Electric-Field Driven Magnetic Domain Wall Motion , 2014, 1411.6798.

[22]  Kaushik Roy,et al.  Hybrid Spintronic-CMOS Spiking Neural Network With On-Chip Learning: Devices, Circuits and Systems , 2015, ArXiv.

[23]  Yan Zhou,et al.  Skyrmion-Electronics: An Overview and Outlook , 2016, Proceedings of the IEEE.

[24]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[25]  Slonczewski Jc,et al.  Conductance and exchange coupling of two ferromagnets separated by a tunneling barrier. , 1989 .

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[27]  Hyun-Woo Lee,et al.  Spin Hall torque magnetometry of Dzyaloshinskii domain walls , 2013, 1308.1432.

[28]  André van Schaik,et al.  AER EAR: A Matched Silicon Cochlea Pair With Address Event Representation Interface , 2005, IEEE Transactions on Circuits and Systems I: Regular Papers.

[29]  T. Moriya.  Anisotropic Superexchange Interaction and Weak Ferromagnetism , 1960 .

[30]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[31]  W. Brown.  Thermal Fluctuations of a Single‐Domain Particle , 1963 .

[32]  Kaushik Roy,et al.  Short-Term Plasticity and Long-Term Potentiation in Magnetic Tunnel Junctions: Towards Volatile Synapses , 2015, ArXiv.

[33]  Shimeng Yu,et al.  Low-Energy Robust Neuromorphic Computation Using Synaptic Devices , 2012, IEEE Transactions on Electron Devices.

[34]  Byong‐Guk Park,et al.  Antiferromagnetic Domain Wall Motion Driven by Spin-Orbit Torques. , 2016, Physical review letters.

[35]  L. Berger,et al.  Low‐field magnetoresistance and domain drag in ferromagnets , 1978 .

[36]  Kaushik Roy,et al.  Spin-Transfer Torque Memories: Devices, Circuits, and Systems , 2016, Proceedings of the IEEE.

[37]  A. Fert,et al.  Spin-polarized current induced switching in Co/Cu/Co pillars , 2001 .

[38]  H. Tanigawa,et al.  Effect of Joule heating in current-driven domain wall motion , 2005 .

[39]  Yan Zhou,et al.  Magnetic skyrmion-based artificial neuron device , 2017, Nanotechnology.

[40]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[41]  S. Parkin,et al.  Domain-wall velocities of up to 750 m s(-1) driven by exchange-coupling torque in synthetic antiferromagnets. , 2015, Nature nanotechnology.

[42]  Ernest J. Torok,et al.  Transition between Bloch and Néel Walls , 1965 .

[43]  G. Beach,et al.  Current-driven dynamics of chiral ferromagnetic domain walls. , 2013, Nature materials.

[44]  J. Slonczewski.  Current-driven excitation of magnetic multilayers , 1996 .

[45]  A. Panchula,et al.  Giant tunnelling magnetoresistance at room temperature with MgO (100) tunnel barriers , 2004, Nature materials.

[46]  Bernard Brezzo,et al.  TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip , 2015, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[47]  Takashi Kimura,et al.  Giant spin-accumulation signal and pure spin-current-induced reversible magnetization switching , 2008 .

[48]  D. Ralph,et al.  Current-induced switching of perpendicularly magnetized magnetic layers using spin torque from the spin Hall effect. , 2012, Physical review letters.

[49]  Kaushik Roy,et al.  Proposal for a Leaky-Integrate-Fire Spiking Neuron Based on Magnetoelectric Switching of Ferromagnets , 2016, IEEE Transactions on Electron Devices.

[50]  Frank van der Velde,et al.  From artificial neural networks to spiking neuron populations and back again , 2001, Neural Networks.

[51]  Jeffrey Bokor,et al.  Deterministic Domain Wall Motion Orthogonal To Current Flow Due To Spin Orbit Torque , 2014, Scientific Reports.

[52]  Ralph,et al.  Current-induced switching of domains in magnetic multilayer devices , 1999, Science.

[53]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[54]  Marc Benayoun,et al.  Avalanches in a Stochastic Model of Spiking Neurons , 2010, PLoS Comput. Biol..

[55]  Kaushik Roy,et al.  Spin Orbit Torque Based Electronic Neuron , 2014, ArXiv.

[56]  Kaushik Roy,et al.  Coupled Spin Torque Nano Oscillators for Low Power Neural Computation , 2015, IEEE Transactions on Magnetics.

[57]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[58]  Kaushik Roy,et al.  On the energy benefits of spiking deep neural networks: A case study , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[59]  D. Querlioz,et al.  Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices , 2013, IEEE Transactions on Nanotechnology.

[60]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[61]  S. Maekawa,et al.  Giant enhancement of spin accumulation and long-distance spin precession in metallic lateral spin valves. , 2011, Nature materials.

[62]  D. Ralph,et al.  Spin-Torque Switching with the Giant Spin Hall Effect of Tantalum , 2012, Science.

[63]  Kaushik Roy,et al.  Spin-Transfer Torque Magnetic neuron for low power neuromorphic computing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[64]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[65]  M. Gajek,et al.  Spin torque switching of 20 nm magnetic tunnel junctions with perpendicular anisotropy , 2012 .

[66]  Luan Tran,et al.  45nm low power CMOS logic compatible embedded STT MRAM utilizing a reverse-connection 1T/1MTJ cell , 2009, 2009 IEEE International Electron Devices Meeting (IEDM).

[67]  Giacomo Indiveri,et al.  A low-power adaptive integrate-and-fire neuron circuit , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[68]  M de Kamps,et al.  From artificial neural networks to spiking neuron populations and back again. , 2001, Neural networks : the official journal of the International Neural Network Society.

[69]  S. Yuasa,et al.  Giant room-temperature magnetoresistance in single-crystal Fe/MgO/Fe magnetic tunnel junctions , 2004, Nature materials.

[70]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[71]  L. Buda-Prejbeanu,et al.  Fast current-induced domain-wall motion controlled by the Rashba effect. , 2011, Nature materials.

[72]  Frank Moss,et al.  Noise enhancement of information transfer in crayfish mechanoreceptors by stochastic resonance , 1993, Nature.

[73]  Kaushik Roy,et al.  STT-SNN: A Spin-Transfer-Torque Based Soft-Limiting Non-Linear Neuron for Low-Power Artificial Neural Networks , 2014, IEEE Transactions on Nanotechnology.

[74]  J. Hirsch.  Spin Hall Effect , 1999, cond-mat/9906160.

[75]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[76]  Kaushik Roy,et al.  Magnetic Tunnel Junction Mimics Stochastic Cortical Spiking Neurons , 2015, Scientific Reports.

[77]  J. W. Brown.  Thermal Fluctuations of a Single-Domain Particle , 1963 .

[78]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[79]  K. Roy,et al.  Spin-Based Neuron Model With Domain-Wall Magnets as Synapse , 2012, IEEE Transactions on Nanotechnology.

[80]  Abhronil Sengupta,et al.  A Vision for All-Spin Neural Networks: A Device to System Perspective , 2016, IEEE Transactions on Circuits and Systems I: Regular Papers.

[81]  Kaushik Roy,et al.  Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning , 2016, Scientific Reports.

[82]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[83]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[84]  Li Sun,et al.  Tuning the properties of magnetic nanowires , 2006, IBM J. Res. Dev..

[85]  Manuel Le Gallo,et al.  Stochastic phase-change neurons. , 2016, Nature nanotechnology.

[86]  Geoffrey S. D. Beach,et al.  Current-induced domain wall motion , 2008 .

[87]  J. Grollier,et al.  Vertical-current-induced domain-wall motion in MgO-based magnetic tunnel junctions with low current densities , 2011, 1102.2106.

[88]  M. Julliere.  Tunneling between ferromagnetic films , 1975 .

[89]  S. Yuasa,et al.  A magnetic synapse: multilevel spin-torque memristor with perpendicular anisotropy , 2016, Scientific Reports.

[90]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[91]  Berger.  Emission of spin waves by a magnetic multilayer traversed by a current. , 1996, Physical review. B, Condensed matter.

[92]  Dmitri E. Nikonov,et al.  Electric-field-induced magnetization reversal in a ferromagnet-multiferroic heterostructure. , 2011, Physical review letters.

[93]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[94]  Damien Querlioz,et al.  Neuromorphic computing with nanoscale spintronic oscillators , 2017, Nature.

[95]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[96]  Kaushik Roy,et al.  Probabilistic Deep Spiking Neural Systems Enabled by Magnetic Tunnel Junction , 2016, IEEE Transactions on Electron Devices.

[97]  Jonathan Z. Sun.  Spin-current interaction with a monodomain magnetic body: A model study , 2000 .

[98]  Kaushik Roy,et al.  Spin-Orbit Torque Induced Spike-Timing Dependent Plasticity , 2014, ArXiv.

[99]  Benjamin Krueger,et al.  Observation of room-temperature magnetic skyrmions and their current-driven dynamics in ultrathin metallic ferromagnets. , 2015, Nature materials.

[100]  I. Dzyaloshinsky.  A thermodynamic theory of “weak” ferromagnetism of antiferromagnetics , 1958 .

[101]  S. J. Martin,et al.  Synaptic plasticity and memory: an evaluation of the hypothesis. , 2000, Annual review of neuroscience.

[102]  Kaushik Roy,et al.  Magnetic tunnel junction enabled all-spin stochastic spiking neural network , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[103]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[104]  Joseph E LeDoux,et al.  Structural plasticity and memory , 2004, Nature Reviews Neuroscience.

[105]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[106]  Romain Brette,et al.  Philosophy of the Spike: Rate-Based vs. Spike-Based Theories of the Brain , 2015, Front. Syst. Neurosci..

[107]  S. Parkin,et al.  Chiral spin torque at magnetic domain walls. , 2013, Nature nanotechnology.

[108]  A. Thomas,et al.  The Memristive Magnetic Tunnel Junction as a Nanoscopic Synapse‐Neuron System , 2012, Advanced materials.

[109]  Atsufumi Hirohata,et al.  Heusler-alloy films for spintronic devices , 2013 .

[110]  Yan Zhou,et al.  Magnetic skyrmion-based synaptic devices , 2016, Nanotechnology.

[111]  Yong Liu,et al.  Specifications of Nanoscale Devices and Circuits for Neuromorphic Computational Systems , 2013, IEEE Transactions on Electron Devices.

[112]  Xiao Cheng,et al.  Nonadiabatic stochastic resonance of a nanomagnet excited by spin torque. , 2010, Physical review letters.

[113]  Y. Hwang,et al.  A 0.24-μm 2.0-V 1T1MTJ 16-kb nonvolatile magnetoresistance RAM with self-reference sensing scheme , 2003, IEEE J. Solid State Circuits.

[114]  A. Brataas,et al.  Spin-orbit torques in action. , 2014, Nature nanotechnology.

[115]  S. Parkin,et al.  Chiral spin torque arising from proximity-induced magnetization , 2014, Nature Communications.

[116]  H. Ohno,et al.  A perpendicular-anisotropy CoFeB-MgO magnetic tunnel junction. , 2010, Nature materials.

[117]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[118]  J. Katine,et al.  Mutual phase-locking of microwave spin torque nano-oscillators , 2005, Nature.

[119]  Slonczewski.  Conductance and exchange coupling of two ferromagnets separated by a tunneling barrier. , 1989, Physical review. B, Condensed matter.

[120]  Jacques-Olivier Klein,et al.  Spin-Transfer Torque Magnetic Memory as a Stochastic Memristive Synapse for Neuromorphic Systems , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[121]  Marc Benayoun,et al.  Emergent Oscillations in Networks of Stochastic Spiking Neurons , 2011, PloS one.

[122]  J. Inoue,et al.  CHAPTER 2 – GMR, TMR and BMR , 2009 .

[123]  J. S. Lee,et al.  Spin-transfer torque generated by a topological insulator , 2014, Nature.

[124]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[125]  Werner Scholz,et al.  Micromagnetic simulation of thermally activated switching in fine particles , 2001 .

[126]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[127]  Geoffrey W. Burr,et al.  Nanoscale electronic synapses using phase change devices , 2013, JETC.

[128]  Kaushik Roy,et al.  Proposal for an All-Spin Artificial Neural Network: Emulating Neural and Synaptic Functionalities Through Domain Wall Motion in Ferromagnets , 2015, IEEE Transactions on Biomedical Circuits and Systems.

[129]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[130]  Johan Åkerman,et al.  Long-range mutual synchronization of spin Hall nano-oscillators , 2016, Nature Physics.

[131]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[132]  W. Regehr,et al.  Short-term synaptic plasticity. , 2002, Annual review of physiology.

[133]  Wolfgang Maass,et al.  STDP enables spiking neurons to detect hidden causes of their inputs , 2009, NIPS.

[134]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[135]  Stuart S. P. Parkin,et al.  Heusler Alloy Films for Spintronic Devices , 2016 .

[136]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[137]  Kaushik Roy,et al.  Toward a spintronic deep learning spiking neural processor , 2016, 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS).