Perspective: A review on memristive hardware for neuromorphic computation

Neuromorphic computation is one of the axes of parallel distributed processing, and memristor-based synaptic weight is considered as a key component of this type of computation. However, the material properties of memristors, including material related physics, are not yet matured. In parallel with memristors, CMOS based Graphics Processing Unit, Field Programmable Gate Array, and Application Specific Integrated Circuit are also being developed as dedicated artificial intelligence (AI) chips for fast computation. Therefore, it is necessary to analyze the competitiveness of the memristor-based neuromorphic device in order to position the memristor in the appropriate position of the future AI ecosystem. In this article, the status of memristor-based neuromorphic computation was analyzed on the basis of papers and patents to identify the competitiveness of the memristor properties by reviewing industrial trends and academic pursuits. In addition, material issues and challenges are discussed for implementing the memristor-based neural processor.

[1]  E. Leobandung,et al.  Capacitor-based Cross-point Array for Analog Neural Network with Record Symmetry and Linearity , 2018, 2018 IEEE Symposium on VLSI Technology.

[2]  Georgios Ch. Sirakoulis,et al.  Experimental Study of Artificial Neural Networks Using a Digital Memristor Simulator , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Shimeng Yu,et al.  Neuro-Inspired Computing With Emerging Nonvolatile Memorys , 2018, Proceedings of the IEEE.

[4]  Hitoshi Kubota,et al.  Neural-like computing with populations of superparamagnetic basis functions , 2016, Nature Communications.

[5]  Majid Ahmadi,et al.  STDP-based unsupervised learning of memristive spiking neural network by Morris-Lecar model , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[6]  Yukihiro Kaneko,et al.  Ferroelectric Artificial Synapses for Recognition of a Multishaded Image , 2014, IEEE Transactions on Electron Devices.

[7]  Hyunsang Hwang,et al.  Analog Synapse Device With 5-b MLC and Improved Data Retention for Neuromorphic System , 2016, IEEE Electron Device Letters.

[8]  Tadahiro Ohmi,et al.  An intelligent MOS transistor featuring gate-level weighted sum and threshold operations , 1991, International Electron Devices Meeting 1991 [Technical Digest].

[9]  Narayan Srinivasa,et al.  Energy-Efficient Neuron, Synapse and STDP Integrated Circuits , 2012, IEEE Transactions on Biomedical Circuits and Systems.

[10]  Pinaki Mazumder,et al.  CMOS and Memristor-Based Neural Network Design for Position Detection , 2012, Proceedings of the IEEE.

[11]  Shimeng Yu,et al.  Synaptic electronics: materials, devices and applications , 2013, Nanotechnology.

[12]  Yunus Babacan,et al.  A spiking and bursting neuron circuit based on memristor , 2016, Neurocomputing.

[13]  Shimeng Yu,et al.  Improving Analog Switching in HfOx-Based Resistive Memory With a Thermal Enhanced Layer , 2017, IEEE Electron Device Letters.

[14]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[15]  Catherine Graves,et al.  Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication , 2016, 2016 53nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[16]  Fırat Kaçar,et al.  Memristor emulator with spike-timing-dependent-plasticity , 2017 .

[17]  Pritish Narayanan,et al.  Neuromorphic computing using non-volatile memory , 2017 .

[18]  Majid Ahmadi,et al.  Modular neuron comprises of memristor-based synapse , 2015, Neural Computing and Applications.

[19]  Leon O. Chua,et al.  Memristor Bridge Synapses , 2012, Proceedings of the IEEE.

[20]  M. Pickett,et al.  A scalable neuristor built with Mott memristors. , 2013, Nature materials.

[21]  Mo Chen,et al.  Three-Dimensional Memristive Hindmarsh-Rose Neuron Model with Hidden Coexisting Asymmetric Behaviors , 2018, Complex..

[22]  U. Ganguly,et al.  PCMO RRAM for Integrate-and-Fire Neuron in Spiking Neural Networks , 2018, IEEE Electron Device Letters.

[23]  Dalibor Biolek,et al.  SPICE modeling of memristive, memcapacitative and meminductive systems , 2009, 2009 European Conference on Circuit Theory and Design.

[24]  Gert Cauwenberghs,et al.  Memristors Empower Spiking Neurons With Stochasticity , 2015, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[25]  Christophe Loyez,et al.  A 4-fJ/Spike Artificial Neuron in 65 nm CMOS Technology , 2017, Front. Neurosci..

[26]  Yukihiro Kaneko,et al.  Supervised Learning Using Spike-Timing-Dependent Plasticity of Memristive Synapses , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Pritish Narayanan,et al.  Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part I: Al/Mo/Pr0.7Ca0.3MnO3 Material Improvements and Device Measurements , 2018, IEEE Journal of the Electron Devices Society.

[28]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[29]  Shimeng Yu,et al.  Ferroelectric FET analog synapse for acceleration of deep neural network training , 2017, 2017 IEEE International Electron Devices Meeting (IEDM).

[30]  E. Eleftheriou,et al.  All-memristive neuromorphic computing with level-tuned neurons , 2016, Nanotechnology.

[31]  Fernando Corinto,et al.  Memristor-based neural circuits , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[32]  Chris Yakopcic,et al.  Multiple memristor read and write circuit for neuromorphic applications , 2011, The 2011 International Joint Conference on Neural Networks.

[33]  Young-Bae Park,et al.  Interpretation of nanoscale conducting paths and their control in nickel oxide (NiO) thin films , 2008 .

[34]  Shinhyun Choi,et al.  SiGe epitaxial memory for neuromorphic computing with reproducible high performance based on engineered dislocations , 2018, Nature Materials.

[35]  Adnan Mehonic,et al.  Emulating the Electrical Activity of the Neuron Using a Silicon Oxide RRAM Cell , 2016, Front. Neurosci..

[36]  Giacomo Indiveri,et al.  A low-power adaptive integrate-and-fire neuron circuit , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[37]  Jing Zhang,et al.  AgInSbTe memristor with gradual resistance tuning , 2013 .

[38]  Lei Deng,et al.  Energy consumption analysis for various memristive networks under different learning strategies , 2016 .

[39]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[40]  D. Seo,et al.  Interpretation of set and reset switching in nickel oxide thin films , 2014 .

[41]  Shinhyun Choi,et al.  Comprehensive physical model of dynamic resistive switching in an oxide memristor. , 2014, ACS nano.

[42]  Majid Ahmadi,et al.  A novel memristor based integrate-and-fire neuron implementation using material implication logic , 2015, 2015 IEEE 28th Canadian Conference on Electrical and Computer Engineering (CCECE).

[43]  Gaurav Mani Khanal,et al.  An a VLSI driving circuit for memristor-based STDP , 2016, 2016 12th Conference on Ph.D. Research in Microelectronics and Electronics (PRIME).

[44]  Chris Yakopcic,et al.  On-chip training of memristor based deep neural networks , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[45]  L. Chua.  Memristor-The missing circuit element , 1971 .

[46]  H. Mulaosmanovic,et al.  Novel ferroelectric FET based synapse for neuromorphic systems , 2017, 2017 Symposium on VLSI Technology.

[47]  Alex Pappachen James,et al.  A Survey of Memristive Threshold Logic Circuits , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[48]  Xiaofeng Liao,et al.  Synchronization and chaos in coupled memristor-based FitzHugh-Nagumo circuits with memristor synapse , 2017 .

[49]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[50]  H. Hwang,et al.  HfZrOx-Based Ferroelectric Synapse Device With 32 Levels of Conductance States for Neuromorphic Applications , 2017, IEEE Electron Device Letters.

[51]  Hao Yu,et al.  Distributed In-Memory Computing on Binary Memristor-Crossbar for Machine Learning , 2017 .

[52]  Pinaki Mazumder,et al.  Learning in Memristor Crossbar-Based Spiking Neural Networks Through Modulation of Weight-Dependent Spike-Timing-Dependent Plasticity , 2018, IEEE Transactions on Nanotechnology.

[53]  Lei Zhang,et al.  Neuromorphic accelerators: A comparison between neuroscience and machine-learning approaches , 2015, 2015 48th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[54]  Sascha Vongehr,et al.  The Missing Memristor has Not been Found , 2015, Scientific Reports.

[55]  Lu You,et al.  Charge trapping-detrapping induced resistive switching in Ba0.7Sr0.3TiO3 , 2012 .

[56]  A S Spinelli,et al.  Memristive neural network for on-line learning and tracking with brain-inspired spike timing dependent plasticity , 2017, Scientific Reports.

[57]  I-Ting Wang,et al.  3D Ta/TaOx/TiO2/Ti synaptic array and linearity tuning of weight update for hardware neural network applications , 2016, Nanotechnology.

[58]  H. Hwang,et al.  Improved Synaptic Behavior Under Identical Pulses Using AlOx/HfO2 Bilayer RRAM Array for Neuromorphic Systems , 2016, IEEE Electron Device Letters.

[59]  Themis Prodromakis,et al.  Analog Memristive Synapse in Spiking Networks Implementing Unsupervised Learning , 2016, Front. Neurosci..

[60]  T. Serrano-Gotarredona,et al.  STDP and STDP variations with memristors for spiking neuromorphic learning systems , 2013, Front. Neurosci..

[61]  Masakazu Aono,et al.  Synaptic plasticity and memory functions achieved in a WO3−x-based nanoionics device by using the principle of atomic switch operation , 2013, Nanotechnology.

[62]  Hyunsang Hwang,et al.  TiOx-Based RRAM Synapse With 64-Levels of Conductance and Symmetric Conductance Change by Adopting a Hybrid Pulse Scheme for Neuromorphic Computing , 2016, IEEE Electron Device Letters.

[63]  Jafar Shamsi,et al.  A Hardware Architecture for Columnar-Organized Memory Based on CMOS Neuron and Memristor Crossbar Arrays , 2018, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[64]  H. Hwang,et al.  Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device , 2011, Nanotechnology.

[65]  Sylvain Saïghi,et al.  Excitatory and Inhibitory Memristive Synapses for Spiking Neural Networks , 2013, 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013).

[66]  J. Grollier,et al.  A ferroelectric memristor. , 2012, Nature materials.

[67]  Farnood Merrikh-Bayat,et al.  Memristor-based perceptron classifier: Increasing complexity and coping with imperfect hardware , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[68]  Ru Huang,et al.  Engineering incremental resistive switching in TaOx based memristors for brain-inspired computing. , 2016, Nanoscale.

[69]  Pinaki Mazumder,et al.  Online Supervised Learning for Hardware-Based Multilayer Spiking Neural Networks Through the Modulation of Weight-Dependent Spike-Timing-Dependent Plasticity , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[70]  Piotr Dudek,et al.  Compact silicon neuron circuit with spiking and bursting behaviour , 2008, Neural Networks.

[71]  Vishal Saxena,et al.  Energy-Efficient CMOS Memristive Synapses for Mixed-Signal Neuromorphic System-on-a-Chip , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).