Experimental analog implementation of Neural Networks on integrated metal-oxide memristive crossbar arrays

It is a well-established fact that Artificial neural networks have superior performance in many information processing tasks. It is also know that by scaling up the size of the network it is possible to have better performance and richer functionality. However, large neural networks are challenging to implement in software and customized hardware are generally required for their practical implementations. Here, we will discuss our group's recent results on the development of such custom hardware circuits, toward the final achievement of hybrid CMOS/memristor circuits, in particular of CMOL variety. We will focus on the experimental and theoretical results for artificial neural networks based on crossbar-integrated metal oxide memristors. The first part will be dedicated to the realization of the first Single Layer Perceptron classifier, while the second part will deal with some preliminary results toward the implementation of Spiking Neural Networks.

[1]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[2]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[3]  O. Richard,et al.  10×10nm2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation , 2011, 2011 International Electron Devices Meeting.

[4]  Timothée Masquelier,et al.  Learning to recognize objects using waves of spikes and Spike Timing-Dependent Plasticity , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[5]  M. Prezioso,et al.  Modeling and implementation of firing-rate neuromorphic-network classifiers with bilayer Pt/Al2O3/TiO2−x/Pt Memristors , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[6]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[7]  Konstantin K. Likharev,et al.  CrossNets: Neuromorphic Hybrid CMOS/Nanoelectronic Networks , 2011 .

[8]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[9]  Farnood Merrikh-Bayat,et al.  Self-Adaptive Spike-Time-Dependent Plasticity of Metal-Oxide Memristors , 2015, Scientific Reports.

[10]  Daniel D. Lee,et al.  Equilibrium properties of temporally asymmetric Hebbian plasticity. , 2000, Physical review letters.

[11]  Wolfram Schiffmann,et al.  Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons , 1994 .

[12]  Wulfram Gerstner,et al.  Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns , 1993, Biological Cybernetics.

[13]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Johannes Schemmel,et al.  Six Networks on a Universal Neuromorphic Computing Substrate , 2012, Front. Neurosci..

[15]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[16]  T. Ciodaro,et al.  Online particle detection with Neural Networks based on topological calorimetry information , 2012 .

[17]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[18]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[19]  Mark C. W. van Rossum,et al.  Stable Hebbian Learning from Spike Timing-Dependent Plasticity , 2000, The Journal of Neuroscience.

[20]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[21]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[22]  Robert P. Sheridan,et al.  Deep Neural Nets as a Method for Quantitative Structure-Activity Relationships , 2015, J. Chem. Inf. Model..

[23]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[24]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[25]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.