Forecasting the outcome of spintronic experiments with Neural Ordinary Differential Equations

Deep learning has an increasing impact to assist research, allowing, for example, the discovery of novel materials. Until now, however, these artificial intelligence techniques have fallen short of discovering the full differential equation of an experimental physical system. Here we show that a dynamical neural network, trained on a minimal amount of data, can predict the behavior of spintronic devices with high accuracy and an extremely efficient simulation time, compared to the micromagnetic simulations that are usually employed to model them. For this purpose, we re-frame the formalism of Neural Ordinary Differential Equations (ODEs) to the constraints of spintronics: few measured outputs, multiple inputs and internal parameters. We demonstrate with Spin-Neural ODEs an acceleration factor over 200 compared to micromagnetic simulations for a complex problem – the simulation of a reservoir computer made of magnetic skyrmions (20 minutes compared to three days). In a second realization, we show that we can predict the noisy response of experimental spintronic nano-oscillators to varying inputs after training Spin-Neural ODEs on five milliseconds of their measured response to different excitations. Spin-Neural ODE is a disruptive tool for developing spintronic applications in complement to micromagnetic simulations, which are time-consuming and cannot fit experiments when noise or imperfections are present. Spin-Neural ODE can also be generalized to other electronic devices involving dynamics.

[1]  Christoph Adelmann,et al.  Opportunities and challenges for spintronics in the microelectronics industry , 2020, Nature Electronics.

[2]  Toshiyuki Yamane,et al.  Recent Advances in Physical Reservoir Computing: A Review , 2018, Neural Networks.

[3]  Damien Querlioz,et al.  Neuromorphic computing with nanoscale spintronic oscillators , 2017, Nature.

[4]  S. Yuasa,et al.  Enhancement of perpendicular magnetic anisotropy and its electric field-induced change through interface engineering in Cr/Fe/MgO , 2016, Scientific Reports.

[5]  R. Goldfarb,et al.  Micromagnetism Applied to Magnetic Nanostructures , 2017 .

[6]  Damien Querlioz,et al.  Physics for neuromorphic computing , 2020, Nature Reviews Physics.

[7]  Jonathan Leliaert,et al.  Tomorrow’s micromagnetic simulations , 2019, Journal of Applied Physics.

[8]  Damien Querlioz,et al.  Vowel recognition with four coupled spin-torque nano-oscillators , 2017, Nature.

[9]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[10]  W. Porod,et al.  Nanoscale neural network using non-linear spin-wave interference , 2020, Nature Communications.

[11]  Y. Wong,et al.  Differentiable Manifolds , 2009 .

[12]  J.-H. Park,et al.  A novel integration of STT-MRAM for on-chip hybrid memory by utilizing non-volatility modulation , 2019, 2019 IEEE International Electron Devices Meeting (IEDM).

[13]  Garrison W. Cottrell,et al.  A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction , 2017, IJCAI.

[14]  Alán Aspuru-Guzik,et al.  Accelerating the discovery of materials for clean energy in the era of smart automation , 2018, Nature Reviews Materials.

[15]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[16]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[17]  M. Jirstrand Algebraic Methods for Modeling and Design in Control , 1996 .

[18]  Kang L. Wang,et al.  Enhancement of voltage-controlled magnetic anisotropy through precise control of Mg insertion thickness at CoFeB|MgO interface , 2017 .

[19]  Supriyo Datta,et al.  Integer factorization using stochastic magnetic tunnel junctions , 2019, Nature.

[20]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[21]  Yan Zhou,et al.  Magnetic skyrmion-based synaptic devices , 2016, Nanotechnology.

[22]  Sun-Ting Tsai,et al.  Learning molecular dynamics with simple language model built upon long short-term memory neural network , 2020, Nature communications.

[23]  F. García-Sánchez,et al.  The design and verification of MuMax3 , 2014, 1406.7635.

[24]  N. Meshkat,et al.  Alternative to Ritt's pseudodivision for finding the input-output equations of multi-output models. , 2012, Mathematical biosciences.

[25]  J. A. Stewart,et al.  Nonlinear Time Series Analysis , 2015 .

[26]  K. Forsman Constructive Commutative Algebra in Nonlinear Control Theory , 1991 .

[27]  Klaus-Robert Müller,et al.  Machine learning of accurate energy-conserving molecular force fields , 2016, Science Advances.

[28]  Boris Livshitz,et al.  FastMag: Fast micromagnetic simulator for complex magnetic structures , 2011 .

[29]  Bogdan Penkovsky Theory and Modeling of Complex Nonlinear Delay Dynamics Applied to Neuromorphic Computing , 2017 .

[30]  CNRS,et al.  Radio-Frequency Multiply-and-Accumulate Operations with Spintronic Synapses , 2020, Physical Review Applied.

[31]  B. Diény,et al.  Review on spintronics: Principles and device applications , 2020, Journal of Magnetism and Magnetic Materials.

[32]  Daniele Pinna,et al.  Reservoir Computing with Random Skyrmion Textures , 2018, Physical Review Applied.

[33]  Simone Finizio,et al.  Skyrmion-based artificial synapses for neuromorphic computing , 2019, Nature Electronics.

[34]  F. Ellinger,et al.  Spintronic based RF components , 2017, 2017 Joint Conference of the European Frequency and Time Forum and IEEE International Frequency Control Symposium (EFTF/IFC).

[35]  Y. Z. Wu,et al.  Magnetic Hamiltonian parameter estimation using deep learning techniques , 2020, Science Advances.

[36]  Julian Vexler,et al.  Deep Neural Networks to Recover Unknown Physical Parameters from Oscillating Time Series , 2021, ArXiv.

[37]  Yan Zhou,et al.  Skyrmion-Electronics: An Overview and Outlook , 2016, Proceedings of the IEEE.

[38]  Robert M. White,et al.  Two-terminal spin–orbit torque magnetoresistive random access memory , 2018, Nature Electronics.

[39]  C. Felser,et al.  The multiple directions of antiferromagnetic spintronics , 2018 .

[40]  Claas Abert,et al.  Micromagnetics and spintronics: models and numerical methods , 2018, The European Physical Journal B.

[41]  F. Takens Detecting strange attractors in turbulence , 1981 .

[42]  J. De Clercq,et al.  Fast micromagnetic simulations on GPU—recent advances made with mumax3 , 2018 .

[43]  Ludovic Denoyer,et al.  Spatio-Temporal Neural Networks for Space-Time Series Forecasting and Relations Discovery , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[44]  A. Fert,et al.  Nucleation, stability and current-induced motion of isolated magnetic skyrmions in nanostructures. , 2013, Nature nanotechnology.

[45]  Gerhard Jakob,et al.  Thermal skyrmion diffusion used in a reshuffler device , 2018, Nature Nanotechnology.

[46]  Patrick Gallinari,et al.  Learning Dynamical Systems from Partial Observations , 2019, ArXiv.

[47]  H. Ohno,et al.  Spintronics based random access memory: a review , 2017 .

[48]  Andrew Gordon Wilson,et al.  Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints , 2020, NeurIPS.

[49]  Yee Whye Teh,et al.  Augmented Neural ODEs , 2019, NeurIPS.

[50]  A. Fert,et al.  Magnetic skyrmions: advances in physics and potential applications , 2017 .