Analog Backpropagation Learning Circuits for Memristive Crossbar Neural Networks

The implementation of backpropagation algorithm using gradient descent operation with analog circuits is an open problem. In this paper, we present the analog learning circuits for realizing backpropagation algorithm for use with neural networks in memristive crossbar arrays. The circuits are simulated in SPICE using TSMC 180nm CMOS process models, and HP memristor models. The gradient descent operations are validated comprehensively using the relevant transfer characteristics and transient response of individual circuit modules.

[1]  Eby G. Friedman,et al.  Memristor-Based Circuit Design for Multilayer Neural Networks , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[2]  Tor Sverre Lande,et al.  Neuromorphic systems engineering: neural networks in silicon , 1998 .

[3]  T. Sejnowski,et al.  Report to the National Science Foundation: WORKSHOP ON NEUROMORPHIC ENGINEERING , 1998 .

[4]  Yves Chauvin,et al.  Backpropagation: theory, architectures, and applications , 1995 .

[5]  Piotr Dudek,et al.  Gradient-descent-based learning in memristive crossbar arrays , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[6]  Anil K. Jain,et al.  Artificial Neural Networks: A Tutorial , 1996, Computer.

[7]  Witali L. Dunin-Barkowski,et al.  An approximate backpropagation learning rule for memristor based neural networks using synaptic plasticity , 2015, Neurocomputing.

[8]  Majid Ahmadi,et al.  Analog Implementation of a Novel Resistive-Type Sigmoidal Neuron , 2012, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[9]  Dalibor Biolek,et al.  Memristor models for SPICE simulation of extremely large memristive networks , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[10]  Farnood Merrikh-Bayat,et al.  Efficient training algorithms for neural networks based on memristive crossbar circuits , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).