A Circuit-Based Learning Architecture for Multilayer Neural Networks With Memristor Bridge Synapses

Memristor-based circuit architecture for multilayer neural networks is proposed. It is a first of its kind demonstrating successful circuit-based learning for multilayer neural network built with memristors. Though back-propagation algorithm is a powerful learning scheme for multilayer neural networks, its hardware implementation is very difficult due to complexities of the neural synapses and the operations involved in the learning algorithm. In this paper, the circuit of a multilayer neural network is designed with memristor bridge synapses and the learning is realized with a simple learning algorithm called Random Weight Change (RWC). Though RWC algorithm requires more iterations than back-propagation algorithm, we show that a circuit-based learning using RWC is two orders faster than its software counterpart. The method to build a multilayer neural network using memristor bridge synapses and a circuit-based learning architecture of RWC algorithm is proposed. Comparison between software-based and memristor circuit-based learning are presented via simulations.

[1]  Leon O. Chua,et al.  Transient Behaviors of Multiple Memristor Circuits Based on Flux Charge Relationship , 2014, Int. J. Bifurc. Chaos.

[2]  K. D. Cantley,et al.  Hebbian Learning in Spiking Neural Networks With Nanocrystalline Silicon TFTs and Memristive Synapses , 2011, IEEE Transactions on Nanotechnology.

[3]  Gregory S. Snider,et al.  Spike-timing-dependent learning in memristive nanodevices , 2008, 2008 IEEE International Symposium on Nanoscale Architectures.

[4]  Leon O. Chua,et al.  Neural Synaptic Weighting With a Pulse-Based Memristor Circuit , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  Leon O. Chua,et al.  Memristor Bridge Synapse-Based Neural Network and Its Learning , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Marwan A. Jabri,et al.  Summed Weight Neuron Perturbation: An O(N) Improvement Over Weight Perturbation , 1992, NIPS.

[7]  S. Tam,et al.  An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses , 1990, International 1989 Joint Conference on Neural Networks.

[8]  Richard A. Chapman,et al.  Neural Learning Circuits Utilizing Nano-Crystalline Silicon Transistors and Memristors , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[10]  Dotan Di Castro,et al.  Hebbian Learning Rules with Memristors , 2013 .

[11]  Jin Liu,et al.  A CMOS feedforward neural-network chip with on-chip parallel learning for oscillation cancellation , 2002, IEEE Trans. Neural Networks.

[12]  Peng Li,et al.  Dynamical Properties and Design Analysis for Nonvolatile Memristor Memories , 2011, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Leon O. Chua,et al.  Composite Behavior of Multiple Memristor Circuits , 2013, IEEE Transactions on Circuits and Systems I: Regular Papers.

[14]  Gert Cauwenberghs,et al.  A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization , 1992, NIPS.

[15]  Yutaka Maeda,et al.  A learning rule of neural networks via simultaneous perturbation and its hardware implementation , 1995, Neural Networks.

[16]  Leon O. Chua.  Resistance switching memories are memristors , 2011 .

[17]  Marwan A. Jabri,et al.  Weight perturbation: an optimal architecture and learning technique for analog VLSI feedforward and recurrent multilayer networks , 1992, IEEE Trans. Neural Networks.

[18]  K. Hirotsu,et al.  An analog neural network chip with random weight change learning algorithm , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[19]  Lawrence D. Jackel,et al.  VLSI implementation of a neural network memory with several hundreds of neurons , 1987 .

[20]  Y. Tamura,et al.  A BiCMOS analog neural network with dynamically updated weights , 1992, 1990 37th IEEE International Conference on Solid-State Circuits.

[21]  Leon O. Chua,et al.  Memristor Bridge Synapses , 2012, Proceedings of the IEEE.

[22]  A. Ayatollahi,et al.  Implementation of biologically plausible spiking neural network models on the memristor crossbar-based CMOS/nano circuits , 2009, 2009 European Conference on Circuit Theory and Design.

[23]  T. G. Habetler,et al.  Identification and control of induction motor stator currents using fast on-line random training of a neural network , 1995, IAS '95. Conference Record of the 1995 IEEE Industry Applications Conference Thirtieth IAS Annual Meeting.

[24]  G. Snider,et al.  Self-organized computation with unreliable, memristive nanodevices , 2007 .