A Plausible Memristor Implementation of Deep Learning Neural Networks

A possible method for hardware implementation of multilayer neural networks with the back-propagation learning algorithm employing memristor cross-bar matrices for weight storage is modeled. The proposed approach offers an efficient way to perform both learning and recognition operations. The solution of several arising problems, such as the representation and multiplication of signals as well as error propagation is proposed.

[1]  Garrett S. Rose,et al.  RRAM-based adaptive neural logic block for implementing non-linearly separable functions in a single layer , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[2]  Leon O. Chua,et al.  Everything You Wish to Know About Memristors But Are Afraid to Ask , 2015 .

[3]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[4]  A. Ayatollahi,et al.  Implementation of biologically plausible spiking neural network models on the memristor crossbar-based CMOS/nano circuits , 2009, 2009 European Conference on Circuit Theory and Design.

[5]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[6]  Leon O. Chua,et al.  A Circuit-Based Learning Architecture for Multilayer Neural Networks With Memristor Bridge Synapses , 2015, IEEE Transactions on Circuits and Systems I: Regular Papers.

[7]  Jason Cong,et al.  Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks , 2015, FPGA.

[8]  Jacques-Olivier Klein,et al.  Robust neural logic block (NLB) based on memristor crossbar array , 2011, 2011 IEEE/ACM International Symposium on Nanoscale Architectures.

[9]  H. Hwang,et al.  An electrically modifiable synapse array of resistive switching memory , 2009, Nanotechnology.

[10]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[11]  Pinaki Mazumder,et al.  CMOS and Memristor-Based Neural Network Design for Position Detection , 2012, Proceedings of the IEEE.

[12]  Leon O. Chua,et al.  Neural Synaptic Weighting With a Pulse-Based Memristor Circuit , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Leon O. Chua,et al.  Memristor Bridge Synapses , 2012, Proceedings of the IEEE.

[14]  Jawar Singh,et al.  Linearly separable pattern classification using memristive crossbar circuits , 2014, Fifteenth International Symposium on Quality Electronic Design.

[15]  Cory Merkel,et al.  Heterogeneous CMOS/memristor hardware neural networks for real-time target classification , 2014, Sensing Technologies + Applications.

[16]  Bernabé Linares-Barranco,et al.  On neuromorphic spiking architectures for asynchronous STDP memristive systems , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[17]  Tarek M. Taha,et al.  Enabling back propagation training of memristor crossbar neuromorphic processors , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[18]  O. Richard,et al.  10×10nm2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation , 2011, 2011 International Electron Devices Meeting.

[19]  Jeyavijayan Rajendran,et al.  An Energy-Efficient Memristive Threshold Logic Circuit , 2012, IEEE Transactions on Computers.

[20]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[21]  Bernard Widrow,et al.  The No-Prop algorithm: A new learning algorithm for multilayer neural networks , 2013, Neural Networks.

[22]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[23]  Damien Querlioz,et al.  Simulation of a memristor-based spiking neural network immune to device variations , 2011, The 2011 International Joint Conference on Neural Networks.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Konstantin K. Likharev,et al.  CrossNets: Neuromorphic Hybrid CMOS/Nanoelectronic Networks , 2011 .

[26]  Huamin Wang,et al.  Pavlov associative memory in a memristive neural network and its circuit implementation , 2016, Neurocomputing.

[27]  Daniel Cownden,et al.  Random feedback weights support learning in deep neural networks , 2014, ArXiv.

[28]  Luca Maria Gambardella,et al.  Better Digit Recognition with a Committee of Simple Neural Nets , 2011, 2011 International Conference on Document Analysis and Recognition.

[29]  Giacomo Indiveri,et al.  Integration of nanoscale memristor synapses in neuromorphic computing architectures , 2013, Nanotechnology.

[30]  Garrett S. Rose,et al.  Memristor-Based Neural Logic Blocks for Nonlinearly Separable Functions , 2013, IEEE Transactions on Computers.

[31]  A. Zenkevich,et al.  Multilevel resistive switching in ternary HfxAl1-xOy oxide with graded Al depth profile , 2013 .

[32]  Piotr Dudek,et al.  Gradient-descent-based learning in memristive crossbar arrays , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[33]  Mika Laiho,et al.  Cellular nanoscale network cell with memristors for local implication logic and synapses , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[34]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[35]  A. Zenkevich,et al.  Resistive switching and synaptic properties of fully atomic layer deposition grown TiN/HfO2/TiN devices , 2015 .

[36]  Avinoam Kolodny,et al.  Memristor-Based Multilayer Neural Networks With Online Gradient Descent Training , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Kyeong-Sik Min,et al.  Self-Adaptive Write Circuit for Low-Power and Variation-Tolerant Memristors , 2010, IEEE Transactions on Nanotechnology.

[38]  Shukai Duan,et al.  Multilayer RTD-memristor-based cellular neural networks for color image processing , 2015, Neurocomputing.

[39]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.