Memristive Extreme Learning Machine: A Neuromorphic Implementation

Neuromorphic computation has been a hot research area over the past few years. Memristor, as one of the neuromorphic computation materials memorizes the conductance value and is able to adapt it according to changing voltages. This paper pioneers a neuromorphic computing paradigm implementation (through memristor) for Extreme Learning Machine (ELM), which is one of most popular machine learning methods. By simulating the biological synapses with memristors and combining the memory property of memristor with high-efficient processing ability in ELM, a three-layer ELM model for classification is constructed. We represent the ELM network weights through memristive conductance values. The conductance values (network weights) are updated through tuning the voltages. Experimental results over the Iris dataset show that the memristor-based ELM achieves the same level performance as the one implemented via traditional software, and exhibits great potential that ELM can be implemented in neromorphic computation paradigms.

[1]  Manuel M. T. Chakravarty,et al.  Accelerating Haskell array codes with multicore GPUs , 2011, DAMP '11.

[2]  Hanan Samet,et al.  A Fast Similarity Join Algorithm Using Graphics Processing Units , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[3]  Sumio Hosaka,et al.  Associative memory realized by a reconfigurable memristive Hopfield neural network , 2015, Nature Communications.

[4]  Jin Wang,et al.  Relational Algebra Algorithms and Data Structures for GPU , 2012 .

[5]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[6]  Mike O'Connor,et al.  Characterizing and evaluating a key-value store application on heterogeneous CPU-GPU systems , 2012, 2012 IEEE International Symposium on Performance Analysis of Systems & Software.

[7]  Chung Lam,et al.  Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array , 2014, Front. Neurosci..

[8]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[9]  Thomas Mikolajick,et al.  Rectifying filamentary resistive switching in ion-exfoliated LiNbO3 thin films , 2016 .

[10]  Byoungil Lee,et al.  Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. , 2012, Nano letters.

[11]  William J. Dally,et al.  The GPU Computing Era , 2010, IEEE Micro.

[12]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[13]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[14]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[15]  Steve Furber,et al.  Neural systems engineering , 2007, Journal of The Royal Society Interface.

[16]  Pedro Trancoso,et al.  Data parallel acceleration of decision support queries using Cell/BE and GPUs , 2009, CF '09.

[17]  Wolfgang Lehner,et al.  GPU-Based Speculative Query Processing for Database Operations , 2010, ADMS@VLDB.

[18]  Massimiliano Di Ventra,et al.  Experimental demonstration of associative memory with memristive neural networks , 2009, Neural Networks.

[19]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  Thomas Sangild Sørensen,et al.  Real-time deformation of detailed geometry based on mappings to a less detailed physical simulation on the GPU , 2005, EGVE'05.

[22]  Kevin Skadron,et al.  Accelerating SQL database operations on a GPU with CUDA , 2010, GPGPU-3.

[23]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[24]  C. Kambhamettu,et al.  GPU implementation of belief propagation using CUDA for cloud tracking and reconstruction , 2008, 2008 IAPR Workshop on Pattern Recognition in Remote Sensing (PRRS 2008).

[25]  Steven Trimberger,et al.  A time-multiplexed FPGA , 1997, Proceedings. The 5th Annual IEEE Symposium on Field-Programmable Custom Computing Machines Cat. No.97TB100186).

[26]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[27]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.