Heterogeneous CMOS/memristor hardware neural networks for real-time target classification

The advent of nanoscale metal-insulator-metal (MIM) structures with memristive properties has given birth to a new generation of hardware neural networks based on CMOS/memristor integration (CMHNNs). The advantage of the CMHNN paradigm compared to a pure CMOS approach lies in the multi-faceted functionality of memristive devices: They can efficiently store neural network configurations (weights and activation function parameters) via non-volatile, quasi-analog resistance states. They also provide high-density interconnects between neurons when integrated into 2-D and 3-D crossbar architectures. In this work, we explore the combination of CMHNN classifiers with manifold learning to reduce the dimensionality of CMHNN inputs. This allows the size of the CMHNN to be reduced significantly (by ≈ 97%). We tested the proposed system using the Caltech101 database and were able to achieve classification accuracies within ≈ 1:5% of those produced by a traditional support vector machine.

[1]  Garrett S. Rose,et al.  RRAM-based adaptive neural logic block for implementing non-linearly separable functions in a single layer , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[2]  Bernabé Linares-Barranco,et al.  On neuromorphic spiking architectures for asynchronous STDP memristive systems , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[3]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[4]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[5]  Damien Querlioz,et al.  Simulation of a memristor-based spiking neural network immune to device variations , 2011, The 2011 International Joint Conference on Neural Networks.

[6]  A. Ayatollahi,et al.  Implementation of biologically plausible spiking neural network models on the memristor crossbar-based CMOS/nano circuits , 2009, 2009 European Conference on Circuit Theory and Design.

[7]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[8]  Ali Ghodsi,et al.  Dimensionality Reduction A Short Tutorial , 2006 .

[9]  Pinaki Mazumder,et al.  CMOS and Memristor-Based Neural Network Design for Position Detection , 2012, Proceedings of the IEEE.

[10]  Christof Teuscher,et al.  Memristor-based reservoir computing , 2012, 2012 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH).

[11]  Leon O. Chua,et al.  Neural Synaptic Weighting With a Pulse-Based Memristor Circuit , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[12]  Andreas E. Savakis,et al.  Manifold based Sparse Representation for robust expression recognition without neutral subtraction , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[13]  Mika Laiho,et al.  Cellular nanoscale network cell with memristors for local implication logic and synapses , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[14]  Dhireesha Kudithipudi,et al.  Periodic activation functions in memristor-based analog neural networks , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[15]  Jeyavijayan Rajendran,et al.  An Energy-Efficient Memristive Threshold Logic Circuit , 2012, IEEE Transactions on Computers.

[16]  Lawrence Cayton,et al.  Algorithms for manifold learning , 2005 .

[17]  Jie Li,et al.  Enhanced supervised locality preserving projections for face recognition , 2011, 2011 International Conference on Machine Learning and Cybernetics.

[18]  Jacques-Olivier Klein,et al.  Robust neural logic block (NLB) based on memristor crossbar array , 2011, 2011 IEEE/ACM International Symposium on Nanoscale Architectures.

[19]  Garrett S. Rose,et al.  Memristor-Based Neural Logic Blocks for Nonlinearly Separable Functions , 2013, IEEE Transactions on Computers.

[20]  Kyeong-Sik Min,et al.  Self-Adaptive Write Circuit for Low-Power and Variation-Tolerant Memristors , 2010, IEEE Transactions on Nanotechnology.

[21]  Bernard Widrow,et al.  The No-Prop algorithm: A new learning algorithm for multilayer neural networks , 2013, Neural Networks.

[22]  Fabien Alibart,et al.  Pattern classification by memristive crossbar circuits using ex situ and in situ training , 2013, Nature Communications.

[23]  Dhireesha Kudithipudi,et al.  A current-mode CMOS/memristor hybrid implementation of an extreme learning machine , 2014, GLSVLSI '14.

[24]  A. Hall,et al.  Adaptive Switching Circuits , 2016 .

[25]  Leon O. Chua,et al.  Memristor Bridge Synapses , 2012, Proceedings of the IEEE.