Neuromorphic hybrid RRAM-CMOS RBM architecture

Restricted Boltzmann Machines (RBMs) offer a key methodology to implement Deep Learning paradigms. This paper presents a novel approach for realizing a hybrid RRAM-CMOS RBM architecture. In our proposed hybrid RBM architecture, HfOx based (filamentary-type switching) RRAM devices are extensively used to implement: (i) Synapses (ii) Internal neuron-state storage and (iii) Stochastic neuron activation function. To validate the proposed scheme we simulated our RBM architecture for classification and reconstruction of hand-written digits on a reduced MNIST dataset of 6000 images. Contrastive-divergence (CD) specially optimized for RRAM devices was used to drive the synaptic weight update mechanism. Total required size of the RRAM matrix in the simulated application is of the order of ~ 0.4 Mb. Peak classification accuracy of 92 %, and an average accuracy of ~ 89 % was obtained over 100 training epochs. Average number of RRAM switching events was ~ 14 million/per epoch.

[1]  Y. Wu,et al.  Variation-aware, reliability-emphasized design and optimization of RRAM using SPICE model , 2015, 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[2]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[3]  Witold Pedrycz,et al.  Contrastive divergence for memristor-based restricted Boltzmann machine , 2015, Engineering applications of artificial intelligence.

[4]  E. Vianello,et al.  Bio-Inspired Stochastic Computing Using Binary CBRAM Synapses , 2013, IEEE Transactions on Electron Devices.

[5]  Fabien Alibart,et al.  OXRAM based ELM architecture for multi-class classification applications , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[6]  Hyunsang Hwang,et al.  Neuromorphic Character Recognition System With Two PCMO Memristors as a Synapse , 2014, IEEE Transactions on Industrial Electronics.

[7]  Jacques-Olivier Klein,et al.  Spin-transfer torque magnetic memory as a stochastic memristive synapse , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[8]  Olivier Bichler,et al.  Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction , 2011, 2011 International Electron Devices Meeting.

[9]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[10]  B. DeSalvo,et al.  Emerging memory technologies: Challenges and opportunities , 2012, Proceedings of Technical Program of 2012 VLSI Technology, System and Application.

[11]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[12]  Bogdan M. Wilamowski,et al.  A VLSI implementation of mixed-signal mode bipolar neuron circuitry , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..