Exploiting the short-term to long-term plasticity transition in memristive nanodevice learning architectures

Memristive nanodevices offer new frontiers for computing systems that unite arithmetic and memory operations on-chip. Here, we explore the integration of electrochemical metallization cell (ECM) nanodevices with tunable filamentary switching in nanoscale learning systems. Such devices offer a natural transition between short-term plasticity (STP) and long-term plasticity (LTP). In this work, we show that this property can be exploited to efficiently solve noisy classification tasks. A single crossbar learning scheme is first introduced and evaluated. Perfect classification is possible only for simple input patterns, within critical timing parameters, and when device variability is weak. To overcome these limitations, a dual-crossbar learning system partly inspired by the extreme learning machine (ELM) approach is then introduced. This approach outperforms a conventional ELM-inspired system when the first layer is imprinted before training and testing, and especially so when variability in device timing evolution is considered: variability is therefore transformed from an issue to a feature. In attempting to classify the MNIST database under the same conditions, conventional ELM obtains 84% classification, the imprinted, uniform device system obtains 88% classification, and the imprinted, variable device system reaches 92% classification. We discuss benefits and drawbacks of both systems in terms of energy, complexity, area imprint, and speed. All these results highlight that tuning and exploiting intrinsic device timing parameters may be of central interest to future bio-inspired approximate computing systems.

[1]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[2]  Jacques-Olivier Klein,et al.  Supervised learning with organic memristor devices and prospects for neural crossbar arrays , 2015, Proceedings of the 2015 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH´15).

[3]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[4]  H. Markram,et al.  Potential for multiple mechanisms, phenomena and algorithms for synaptic plasticity at single synapses , 1998, Neuropharmacology.

[5]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[6]  W. Abraham.  Metaplasticity: tuning synapses and networks for plasticity , 2008, Nature Reviews Neuroscience.

[7]  S. Jo,et al.  3D-stackable crossbar resistive memory based on Field Assisted Superlinear Threshold (FAST) selector , 2014, 2014 IEEE International Electron Devices Meeting.

[8]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[9]  André van Schaik,et al.  Explicit Computation of Input Weights in Extreme Learning Machines , 2014, ArXiv.

[10]  Dominique Vuillaume,et al.  Filamentary switching: synaptic plasticity through device volatility. , 2015, ACS nano.

[11]  Jacques-Olivier Klein,et al.  Spin-transfer torque magnetic memory as a stochastic memristive synapse , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Zhaohao Wang,et al.  On-chip supervised learning rule for ultra high density neural crossbar using memristor for synapse and neuron , 2014, 2014 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[14]  Manan Suri,et al.  Exploiting Intrinsic Variability of Filamentary Resistive Memory for Extreme Learning Machine Architectures , 2015, IEEE Transactions on Nanotechnology.

[15]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.

[16]  Wei Lu,et al.  Short-term Memory to Long-term Memory Transition in a Nanoscale Memristor , 2022 .

[17]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[18]  Y. Liu,et al.  Synaptic Learning and Memory Functions Achieved Using Oxygen Ion Migration/Diffusion in an Amorphous InGaZnO Memristor , 2012 .

[19]  T. Sejnowski,et al.  Nanoconnectomic upper bound on the variability of synaptic plasticity , 2015, eLife.

[20]  Avinoam Kolodny,et al.  Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Mark D. McDonnell,et al.  Fast, Simple and Accurate Handwritten Digit Classification by Training Shallow Neural Network Classifiers with the ‘Extreme Learning Machine’ Algorithm , 2015, PloS one.

[22]  C. Teuscher,et al.  Volatile memristive devices as short-term memory in a neuromorphic learning architecture , 2014, 2014 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[23]  Damien Querlioz,et al.  Bioinspired Programming of Memory Devices for Implementing an Inference Engine , 2015, Proceedings of the IEEE.

[24]  Jacques-Olivier Klein,et al.  On-Chip Universal Supervised Learning Methods for Neuro-Inspired Block of Memristive Nanodevices , 2015, ACM J. Emerg. Technol. Comput. Syst..

[25]  Zhaohao Wang,et al.  Ultrahigh Density Memristor Neural Crossbar for On-Chip Supervised Learning , 2015, IEEE Transactions on Nanotechnology.

[26]  André van Schaik,et al.  Online and adaptive pseudoinverse solutions for ELM weights , 2015, Neurocomputing.

[27]  Dmitri B. Strukov,et al.  Low area overhead in-situ training approach for memristor-based classifier , 2015, Proceedings of the 2015 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH´15).

[28]  Fabien Alibart,et al.  Plasticity in memristive devices for spiking neural networks , 2015, Front. Neurosci..