Volatile memristive devices as short-term memory in a neuromorphic learning architecture

Image classification with feed-forward neural networks typically assumes the application of input images as single column vectors, which leads to a large number of required input neurons as well as large synaptic arrays connecting individual neural layers. In this paper we show how a class of memristive devices can be used as non-linear, leaky integrators that extend regular feed-forward neural networks with short-term memory. By trading space for time, our novel architecture allows to reduce the number of neurons by a factor of 3 and the number of synapses up to 15 times on the MNIST data set compared to previously reported results. Furthermore, the results indicate that less neurons and synapses also leads to a reduced learning complexity. With memristive devices functioning as dynamic processing elements, our findings advocate for a diverse use of memristive devices that would allow to build more area-efficient hardware by exploiting more than just their non-volatile memory property.

[1]  W. Lu,et al.  High-density Crossbar Arrays Based on a Si Memristive System , 2008 .

[2]  Ali Khiat,et al.  Resistive switching of oxygen enhanced TiO2 thin-film devices , 2013 .

[3]  Jacques-Olivier Klein,et al.  Bioinspired networks with nanoscale memristive devices that combine the unsupervised and supervised learning approaches , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[4]  Damien Querlioz,et al.  Simulation of a memristor-based spiking neural network immune to device variations , 2011, The 2011 International Joint Conference on Neural Networks.

[5]  Wolfgang Maass,et al.  Liquid State Machines: Motivation, Theory, and Applications , 2010 .

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  T. Serrano-Gotarredona,et al.  STDP and STDP variations with memristors for spiking neuromorphic learning systems , 2013, Front. Neurosci..

[8]  Wei Lu,et al.  Interference and memory capacity effects in memristive systems , 2013 .

[9]  Henry Markram,et al.  The "Liquid Computer": A Novel Strategy for Real-Time Computing on Time Series , 2002 .

[10]  Siddharth Gaba,et al.  Synaptic behaviors and modeling of a metal oxide memristive device , 2011 .

[11]  D. Strukov,et al.  CMOL: Devices, Circuits, and Architectures , 2006 .

[12]  Yuchao Yang,et al.  Building Neuromorphic Circuits with Memristive Devices , 2013, IEEE Circuits and Systems Magazine.

[13]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[14]  Ligang Gao,et al.  High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm , 2011, Nanotechnology.

[15]  IEEE/ACM International Symposium on Nanoscale Architectures, NANOARCH 2016, Beijing, China, July 18-20, 2016 , 2016, NANOARCH.