GST-memristor-based online learning neural networks

Abstract At present, it is an urgent issue to effectively train artificial neural network (ANN), especially when the data is large. Online learning has been used to solve the problem, most of which is based on least mean square (LMS). However, it is inefficient to implement the LMS on conventional digital hardware, because of the physical separation between the memory arrays and arithmetic module. To solve this problem, CMOS has been utilized. However, it costs too many powers and areas while designing CMOS synapses in the very large scale integrated (VLSI) circuit. As a novel device, memristor is believed to overcome this shortcoming as memristors could be utilized to store the weights which could be changed by a voltage pulse. The filamentary bipolar memristive switching in Ge2Sb2Te5 (GST) has been proved to be an ideal choice for memristive materials. And it has two states—amorphous and crystalline, which can be changed by DC sweep. In this paper, we consider an artificial synapse which includes a GST-memristor and two MOSFET transistors (p-type and n-type). A number of artificial synapses are employed to form a circuit which is expected to consume 2 − 8 % of the area compared to CMOS-only circuit. And the accuracy is about 80%, which is good enough in realistic diagnosis and has good robustness with noise.

[1]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[2]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[3]  Fernando Corinto,et al.  A Boundary Condition-Based Approach to the Modeling of Memristor Nanostructures , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[4]  Chuandong Li,et al.  Sandwich control systems with impulse time windows , 2017, Int. J. Mach. Learn. Cybern..

[5]  Avinoam Kolodny,et al.  Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Tingwen Huang,et al.  Passivity of Directed and Undirected Complex Dynamical Networks With Adaptive Coupling Weights , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Junzhi Yu,et al.  Linear impulsive control system with impulse time windows , 2017 .

[8]  Amos Omondi.  Neurocomputers: A Dead End? , 2000, Int. J. Neural Syst..

[9]  Dacheng Tao,et al.  Deep Neural Network for Structural Prediction and Lane Detection in Traffic Scene , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[10]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Massimiliano Versace,et al.  The brain of a new machine , 2010, IEEE Spectrum.

[12]  Lihong Li,et al.  Unbiased online active learning in data streams , 2011, KDD.

[13]  X. Miao,et al.  Ultrafast Synaptic Events in a Chalcogenide Memristor , 2013, Scientific Reports.

[14]  Jun Wang,et al.  Passivity and Passification of Memristor-Based Recurrent Neural Networks With Time-Varying Delays , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[15]  L. Chua.  Memristor-The missing circuit element , 1971 .

[16]  Jeyavijayan Rajendran,et al.  Stochastic Gradient Descent Inspired Training Technique for a CMOS/Nano Memristive Trainable Threshold Gate Array , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[17]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[18]  Jun Wang,et al.  Global Exponential Synchronization of Multiple Memristive Neural Networks With Time Delay via Nonlinear Coupling , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[20]  Chuandong Li,et al.  Robust Exponential Stability of Uncertain Delayed Neural Networks With Stochastic Perturbation and Impulse Effects , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[21]  T. Makino.  A Discrete-Event Neural Network Simulator for General Neuron Models , 2003, Neural Computing & Applications.

[22]  D. Querlioz,et al.  Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices , 2013, IEEE Transactions on Nanotechnology.

[23]  Jun Wang,et al.  Robust Synchronization of Multiple Memristive Neural Networks With Uncertain Parameters via Nonlinear Coupling , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[24]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[25]  Chuandong Li,et al.  Periodically multiple state-jumps impulsive control systems with impulse time windows , 2016, Neurocomputing.

[26]  Byoung Hun Lee,et al.  Excellent resistive switching in nitrogen-doped Ge2Sb2Te5 devices for field-programmable gate array configurations , 2011 .

[27]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[28]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[29]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[30]  Tingwen Huang,et al.  Passivity-based synchronization of a class of complex dynamical networks with time-varying delay , 2015, Autom..

[31]  Zhigang Zeng,et al.  Adaptive Neural-Fuzzy Sliding-Mode Fault-Tolerant Control for Uncertain Nonlinear Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[32]  Uri C. Weiser,et al.  TEAM: ThrEshold Adaptive Memristor Model , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[33]  Zhigang Zeng,et al.  A short-term power load forecasting model based on the generalized regression neural network with decreasing step fruit fly optimization algorithm , 2017, Neurocomputing.

[34]  L.O. Chua,et al.  Memristive devices and systems , 1976, Proceedings of the IEEE.

[35]  Zhigang Zeng,et al.  Synchronization of Switched Neural Networks With Communication Delays via the Event-Triggered Control , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[37]  Dong Wang,et al.  Complex Learning in Bio-plausible Memristive Networks , 2015, Scientific Reports.

[38]  Uri C. Weiser,et al.  Memristor-Based Multithreading , 2014, IEEE Computer Architecture Letters.

[39]  Tingwen Huang,et al.  Alternate control delayed systems , 2015 .

[40]  Léon Bottou,et al.  The Tradeoffs of Large Scale Learning , 2007, NIPS.

[41]  Tingwen Huang,et al.  Alternate control systems , 2014 .

[42]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[43]  Sander M. Bohte,et al.  Computing with Spiking Neuron Networks , 2012, Handbook of Natural Computing.

[44]  Zhigang Zeng,et al.  Circuit design and exponential stabilization of memristive neural networks , 2015, Neural Networks.

[45]  Wu Jigang,et al.  Pinning Control for Synchronization of Coupled Reaction-Diffusion Neural Networks With Directed Topologies , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[46]  Jingzhe Wang.  Some new results for the (p,q)p , 2014 .

[47]  Huai-Ning Wu,et al.  Pinning Control Strategies for Synchronization of Linearly Coupled Neural Networks With Reaction–Diffusion Terms , 2016, IEEE Transactions on Neural Networks and Learning Systems.