Practical gradient-descent for memristive crossbars

This paper discusses implementations of gradientdescent based learning algorithms on memristive crossbar arrays. The Unregulated Step Descent (USD) is described as a practical algorithm for feed-forward on-line training of large crossbar arrays. It allows fast feed-forward fully parallel on-line hardware based learning, without requiring accurate models of the memristor behaviour and precise control of the programming pulses. The effect of device parameters, training parameters, and device variability on the learning performance of crossbar arrays trained using the USD algorithm has been studied via simulations.

[1]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[2]  I. Ial,et al.  Nature Communications , 2010, Nature Cell Biology.

[3]  R. Rosenfeld.  Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[4]  Fred Rothganger,et al.  Training neural hardware with noisy components , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[5]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[6]  Cheng-Fu Yang,et al.  Prepare dispersed CIS nano-scale particles and spray coating CIS absorber layers using nano-scale precursors , 2014, Nanoscale Research Letters.

[7]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[8]  Farnood Merrikh-Bayat,et al.  Efficient training algorithms for neural networks based on memristive crossbar circuits , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[9]  Chris Yakopcic,et al.  Memristor based neuromorphic circuit for ex-situ training of multi-layer neural network algorithms , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[10]  Piotr Dudek,et al.  Gradient-descent-based learning in memristive crossbar arrays , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[11]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[12]  D. Koshland.  Frontiers in neuroscience. , 1988, Science.

[13]  G. Schmid.  The Nature of Nanotechnology , 2010 .

[14]  Yiran Chen,et al.  Memristor Crossbar-Based Neuromorphic Computing System: A Case Study , 2014, IEEE Transactions on Neural Networks and Learning Systems.