Transiently chaotic simulated annealing based on intrinsic nonlinearity of memristors for efficient solution of optimization problems

This work shows a memristive optimizer based on transient chaos, which is able to solve optimization problems efficiently. Optimization problems are ubiquitous in scientific research, engineering, and daily lives. However, solving a complex optimization problem often requires excessive computing resource and time and faces challenges in easily getting trapped into local optima. Here, we propose a memristive optimizer hardware based on a Hopfield network, which introduces transient chaos to simulated annealing in aid of jumping out of the local optima while ensuring convergence. A single memristor crossbar is used to store the weight parameters of a fully connected Hopfield network and adjust the network dynamics in situ. Furthermore, we harness the intrinsic nonlinearity of memristors within the crossbar to implement an efficient and simplified annealing process for the optimization. Solutions of continuous function optimizations on sphere function and Matyas function as well as combinatorial optimization on Max-cut problem are experimentally demonstrated, indicating great potential of the transiently chaotic memristive network in solving optimization problems in general.

[1]  Qing Wu,et al.  Long short-term memory networks in memristor crossbar arrays , 2018, Nature Machine Intelligence.

[2]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[3]  Mohammed A. Zidan,et al.  Hardware Acceleration of Simulated Annealing of Spin Glass by RRAM Crossbar Array , 2018, 2018 IEEE International Electron Devices Meeting (IEDM).

[4]  S. Reuveni,et al.  Multisite phosphorylation drives phenotypic variation in (p)ppGpp synthetase-dependent antibiotic tolerance , 2019, Nature Communications.

[5]  C. Teuscher,et al.  Modeling and Experimental Demonstration of a Hopfield Network Analog-to-Digital Converter with Hybrid CMOS/Memristor Circuits , 2015, Front. Neurosci..

[6]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[7]  Daniele Ielmini,et al.  Solving matrix equations in one step with cross-point resistive arrays , 2019, Proceedings of the National Academy of Sciences.

[8]  M. R. Mahmoodi,et al.  Versatile stochastic dot product circuits based on nonvolatile memories for high performance neurocomputing and neurooptimization , 2019, Nature Communications.

[9]  KozyrakisChristos,et al.  Understanding sources of ineffciency in general-purpose chips , 2010 .

[10]  Martin Grötschel,et al.  An Application of Combinatorial Optimization to Statistical Physics and Circuit Layout Design , 1988, Oper. Res..

[11]  Yuchao Yang,et al.  Probing memristive switching in nanoionic devices , 2018 .

[12]  Ted K. Ralphs,et al.  Integer and Combinatorial Optimization , 2013 .

[13]  Jiaming Zhang,et al.  Analogue signal and image processing with large memristor crossbars , 2017, Nature Electronics.

[14]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[15]  Catherine E. Graves,et al.  Memristor‐Based Analog Computation and Neural Network Classification with a Dot Product Engine , 2018, Advanced materials.

[16]  Huan Liu,et al.  Optimization of non-linear conductance modulation based on metal oxide memristors , 2018, Nanotechnology Reviews.

[17]  Bing Chen,et al.  A general memristor-based partial differential equation solver , 2018, Nature Electronics.

[18]  Yuchao Yang,et al.  Probing nanoscale oxygen ion motion in memristive systems , 2017, Nature Communications.

[19]  Mark Horowitz,et al.  1.1 Computing's energy problem (and what we can do about it) , 2014, 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC).

[20]  Hyunsang Hwang,et al.  Improved Synapse Device With MLC and Conductance Linearity Using Quantized Conduction for Neuromorphic Systems , 2018, IEEE Electron Device Letters.

[21]  Christoforos E. Kozyrakis,et al.  Understanding sources of inefficiency in general-purpose chips , 2010, ISCA.

[22]  Chung Lam,et al.  Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array , 2014, Front. Neurosci..

[23]  H. Nili,et al.  An Analog Neuro-Optimizer with Adaptable Annealing Based on 64×64 0T1R Crossbar Circuit , 2019, 2019 IEEE International Electron Devices Meeting (IEDM).

[24]  L. Chua.  Memristor-The missing circuit element , 1971 .

[25]  Qing Wu,et al.  In situ training of feed-forward and recurrent convolutional memristor networks , 2019, Nature Machine Intelligence.

[26]  Farnood Merrikh-Bayat,et al.  Digital-to-analog and analog-to-digital conversion with metal oxide memristors for ultra-low power computing , 2013, 2013 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[27]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[28]  H. Hwang,et al.  Improved Synaptic Behavior Under Identical Pulses Using AlOx/HfO2 Bilayer RRAM Array for Neuromorphic Systems , 2016, IEEE Electron Device Letters.

[29]  Peng Lin,et al.  Reinforcement learning with analogue memristor arrays , 2019, Nature Electronics.

[30]  Sumio Hosaka,et al.  Associative memory realized by a reconfigurable memristive Hopfield neural network , 2015, Nature Communications.

[31]  Shimeng Yu,et al.  Harnessing Intrinsic Noise in Memristor Hopfield Neural Networks for Combinatorial Optimization , 2019, ArXiv.

[32]  Yuchao Yang,et al.  Memristive Devices and Networks for Brain‐Inspired Computing , 2019, physica status solidi (RRL) – Rapid Research Letters.

[33]  Shimeng Yu,et al.  Improving Analog Switching in HfOx-Based Resistive Memory With a Thermal Enhanced Layer , 2017, IEEE Electron Device Letters.

[34]  Kazuyuki Aihara,et al.  Chaotic simulated annealing by a neural network model with transient chaos , 1995, Neural Networks.

[35]  Kazuyuki Aihara,et al.  Adaptive annealing for chaotic optimization , 1996 .

[36]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[37]  Cheng-Chieh Chang,et al.  Pseudo-exponential function for MOSFETs in saturation , 2000 .

[38]  Bin Gao,et al.  Associative Memory for Image Recovery with a High‐Performance Memristor Array , 2019, Advanced Functional Materials.

[39]  Ru Huang,et al.  A comprehensive review on emerging artificial neuromorphic devices , 2020, Applied Physics Reviews.

[40]  Kang L. Wang,et al.  Resistive switching materials for information processing , 2020, Nature Reviews Materials.

[41]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.