Gradient-descent-based learning in memristive crossbar arrays

This paper describes techniques to implement gradient-descent-based machine learning algorithms on crossbar arrays made of memristors or other analog memory devices. We introduce the Unregulated Step Descent (USD) algorithm, which is an approximation of the steepest descent algorithm, and discuss how it addresses various hardware implementation issues. We discuss the effect of device parameters and their variability on performance of the algorithm by using artificially generated and real-world datasets. In addition to providing insights on the effect of device parameters on learning, we illustrate how the USD algorithm partially offsets the effect of device variability. Finally, we discuss how the USD algorithm can be implemented in crossbar arrays using a simple 4-phase training scheme. The method allows parallel update of crossbar memory elements and reduces the hardware cost and complexity of the training architecture significantly.

[1]  Gert Cauwenberghs,et al.  A Fast Stochastic Error-Descent Algorithm for Supervised Learning and Optimization , 1992, NIPS.

[2]  Bernabé Linares-Barranco,et al.  On Spike-Timing-Dependent-Plasticity, Memristive Devices, and Building a Self-Learning Visual Cortex , 2011, Front. Neurosci..

[3]  Mika Laiho,et al.  Two memristors suffice to compute all Boolean functions (Organic and inorganic circuits and devices) , 2010 .

[4]  Fernando Corinto,et al.  A Boundary Condition-Based Approach to the Modeling of Memristor Nanostructures , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[5]  Y. So,et al.  A Tutorial on Logistic Regression , 1996 .

[6]  Shimeng Yu,et al.  On the Switching Parameter Variation of Metal-Oxide RRAM—Part I: Physical Modeling and Simulation Methodology , 2012, IEEE Transactions on Electron Devices.

[7]  Ali Khiat,et al.  Temporal processing with volatile memristors , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[8]  Dhireesha Kudithipudi,et al.  Reconfigurable N-level memristor memory design , 2011, The 2011 International Joint Conference on Neural Networks.

[9]  G. Snider,et al.  Self-organized computation with unreliable, memristive nanodevices , 2007 .

[10]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[11]  Damien Querlioz,et al.  Simulation of a memristor-based spiking neural network immune to device variations , 2011, The 2011 International Joint Conference on Neural Networks.

[12]  J. Yang,et al.  Feedback write scheme for memristive switching devices , 2011 .

[13]  Garrett S. Rose,et al.  A read-monitored write circuit for 1T1M multi-level memristor memories , 2011, 2011 IEEE International Symposium of Circuits and Systems (ISCAS).

[14]  M. Pickett,et al.  Lognormal switching times for titanium dioxide bipolar memristors: origin and resolution , 2011, Nanotechnology.

[15]  F. Corinto,et al.  Memristor Model Comparison , 2013, IEEE Circuits and Systems Magazine.

[16]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[17]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[18]  Gregory S. Snider,et al.  Spike-timing-dependent learning in memristive nanodevices , 2008, 2008 IEEE International Symposium on Nanoscale Architectures.

[19]  Daniel Krebs,et al.  Crystal growth within a phase change memory cell , 2014, Nature Communications.

[20]  Ron Meir,et al.  A Parallel Gradient Descent Method for Learning in Analog VLSI Neural Networks , 1992, NIPS.

[21]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[22]  Bernabé Linares-Barranco,et al.  Memristance can explain Spike-Time-Dependent-Plasticity in Neural Synapses , 2009 .

[23]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[24]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.