Improved Hopfield Network Optimization Using Manufacturable Three-Terminal Electronic Synapses

We illustrate novel optimization techniques via simulations for Hopfield networks constructed from manufacturable three-terminal Silicon-Oxide-Nitride-Oxide-Silicon (SONOS) synaptic circuit elements. We first present a computationally-light, memristor-based, highly accurate static compact model for the SONOS synapses used in our simulations. We then show how to exploit analog errors in programming resistances and current leakage, and the continuous tunability of the SONOS synapses to enable transient chaotic group dynamics, to accelerate the convergence of a Hopfield network. We project improvements in energy consumption and time to solution relative to existing CPUs and GPUs by at least 4 orders of magnitude, and also exceed the projected performance of two-terminal memristor-based crossbars in addition to a 100-fold increase in error-resilient array size (i.e. problem size).

[1]  Raymond Beausoleil,et al.  Power-efficient combinatorial optimization using intrinsic noise in memristor Hopfield neural networks , 2020, Nature Electronics.

[2]  Kazuyuki Aihara,et al.  Chaotic simulated annealing by a neural network model with transient chaos , 1995, Neural Networks.

[3]  J. Yang,et al.  High switching endurance in TaOx memristive devices , 2010 .

[4]  R. Williams,et al.  Separation of current density and electric field domains caused by nonlinear electronic instabilities , 2018, Nature Communications.

[5]  Catherine E. Graves,et al.  Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine , 2018, Advanced materials.

[6]  J. Yang,et al.  Memristor crossbar arrays with 6-nm half-pitch and 2-nm critical dimension , 2018, Nature Nanotechnology.

[7]  Ziwen Wang,et al.  Third-order nanocircuit elements for neuromorphic engineering , 2020, Nature.

[8]  Miguel Muñoz Rojo,et al.  Size scaling, dynamics, and electro-thermal bifurcation of VO2 Mott oscillators. , 2020, 2012.07943.

[9]  J. Yang,et al.  Switching dynamics in titanium dioxide memristive devices , 2009 .

[10]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[11]  E. Vianello,et al.  In situ learning using intrinsic memristor variability via Markov chain Monte Carlo sampling , 2021 .

[12]  Henry Leung,et al.  A study of the transiently chaotic neural network for combinatorial optimization , 2002 .

[13]  Eric Pop,et al.  Fast Spiking of a Mott VO2-Carbon Nanotube Composite Device. , 2019, Nano letters.

[14]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[15]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[16]  J. Fontanari,et al.  Stochastic versus deterministic update in simulated annealing , 1990 .

[17]  Matthew J. Marinella,et al.  Using Floating-Gate Memory to Train Ideal Accuracy Neural Networks , 2019, IEEE Journal on Exploratory Solid-State Computational Devices and Circuits.

[18]  M. R. Mahmoodi,et al.  Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization , 2019, Nature Communications.

[19]  John Paul Strachan,et al.  Chaotic dynamics in nanoscale NbO2 Mott memristors for analogue computing , 2017, Nature.