Memristor Bridge Synapse-Based Neural Network and Its Learning

Analog hardware architecture of a memristor bridge synapse-based multilayer neural network and its learning scheme is proposed. The use of memristor bridge synapse in the proposed architecture solves one of the major problems, regarding nonvolatile weight storage in analog neural network implementations. To compensate for the spatial nonuniformity and nonideal response of the memristor bridge synapse, a modified chip-in-the-loop learning scheme suitable for the proposed neural network architecture is also proposed. In the proposed method, the initial learning is conducted in software, and the behavior of the software-trained network is learned by the hardware network by learning each of the single-layered neurons of the network independently. The forward calculation of the single-layered neuron learning is implemented on circuit hardware, and followed by a weight updating phase assisted by a host computer. Unlike conventional chip-in-the-loop learning, the need for the readout of synaptic weights for calculating weight updates in each epoch is eliminated by virtue of the memristor bridge synapse and the proposed learning scheme. The hardware architecture along with the successful implementation of proposed learning on a three-bit parity network, and on a car detection network is also presented.

[1]  L. Chua.  Memristor-The missing circuit element , 1971 .

[2]  Lawrence D. Jackel,et al.  VLSI implementation of a neural network memory with several hundreds of neurons , 1987 .

[3]  S. Tam,et al.  An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses , 1990, International 1989 Joint Conference on Neural Networks.

[4]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[5]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[6]  Y. Tamura,et al.  A BiCMOS analog neural network with dynamically updated weights , 1992, 1990 37th IEEE International Conference on Solid-State Circuits.

[7]  B. Gupta,et al.  Learning on an analog VLSI neural network chip , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[8]  Marwan A. Jabri,et al.  Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks , 1992, IEEE Trans. Neural Networks.

[9]  Tetsuro Itakura,et al.  Neuro chips with on-chip back-propagation and/or Hebbian learning , 1992 .

[10]  Jerzy B. Lont.  Analog CMOS implementation of a multi-layer perceptron with nonlinear synapses , 1992, IEEE Trans. Neural Networks.

[11]  K. Hirotsu,et al.  An analog neural network chip with random weight change learning algorithm , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[12]  Yutaka Maeda,et al.  A learning rule of neural networks via simultaneous perturbation and its hardware implementation , 1995, Neural Networks.

[13]  Frank L. Lewis,et al.  Multilayer neural-net robot controller with guaranteed tracking performance , 1996, IEEE Trans. Neural Networks.

[14]  Ronald S. Gyurcsik,et al.  Toward a general-purpose analog VLSI neural network with on-chip learning , 1997, IEEE Trans. Neural Networks.

[15]  Emile Fiesler,et al.  Neural Network Adaptations to Hardware Implementations , 1997 .

[16]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Frank L. Lewis,et al.  Control of a nonholonomic mobile robot using neural networks , 1998, IEEE Trans. Neural Networks.

[18]  Sorin Draghici,et al.  Neural Networks in Analog Hardware - Design and Implementation Issues , 2000, Int. J. Neural Syst..

[19]  Leonardo Maria Reyneri.  Implementation issues of neuro-fuzzy hardware: going toward HW/SW codesign , 2003, IEEE Trans. Neural Networks.

[20]  Fernando Morgado Dias,et al.  Artificial neural networks: a review of commercial hardware , 2004, Eng. Appl. Artif. Intell..

[21]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[22]  G. Snider,et al.  Self-organized computation with unreliable, memristive nanodevices , 2007 .

[23]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[24]  Stephen J. Wolf,et al.  The elusive memristor: properties of basic electrical circuits , 2008, 0807.3994.

[25]  A. Ayatollahi,et al.  Implementation of biologically plausible spiking neural network models on the memristor crossbar-based CMOS/nano circuits , 2009, 2009 European Conference on Circuit Theory and Design.

[26]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[27]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[28]  D. Hammerstrom.  A Survey of Bio‐Inspired and Other Alternative Architectures , 2010 .

[29]  Tülay Yildirim,et al.  Conic Section Function Neural Network Circuitry for Offline Signature Recognition , 2010, IEEE Transactions on Neural Networks.

[30]  Peng Li,et al.  Dynamical Properties and Design Analysis for Nonvolatile Memristor Memories , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[31]  Richard A. Chapman,et al.  Neural Learning Circuits Utilizing Nano-Crystalline Silicon Transistors and Memristors , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Leon O. Chua,et al.  Memristor Bridge Synapses , 2012, Proceedings of the IEEE.

[33]  Leon O. Chua,et al.  Neural Synaptic Weighting With a Pulse-Based Memristor Circuit , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.