Complementary Metal‐Oxide Semiconductor and Memristive Hardware for Neuromorphic Computing

The ever‐increasing processing power demands of digital computers cannot continue to be fulfilled indefinitely unless there is a paradigm shift in computing. Neuromorphic computing, which takes inspiration from the highly parallel, low‐power, high‐speed, and noise‐tolerant computing capabilities of the brain, may provide such a shift. Many researchers from across academia and industry have been studying materials, devices, circuits, and systems, to implement some of the functions of networks of neurons and synapses to develop neuromorphic computing platforms. These platforms are being designed using various hardware technologies, including the well‐established complementary metal‐oxide semiconductor (CMOS), and emerging memristive technologies such as SiOx‐based memristors. Herein, recent progress in CMOS, SiOx‐based memristive, and mixed CMOS‐memristive hardware for neuromorphic systems is highlighted. New and published results from various devices are provided that are developed to replicate selected functions of neurons, synapses, and simple spiking networks. It is shown that the CMOS and memristive devices are assembled in different neuromorphic learning platforms to perform simple cognitive tasks such as classification of spike rate‐based patterns or handwritten digits. Herein, it is envisioned that what is demonstrated is useful to the unconventional computing research community by providing insights into advances in neuromorphic hardware technologies.

[1]  Giacomo Indiveri,et al.  An Event-Based Neural Network Architecture With an Asynchronous Programmable Synaptic Memory , 2014, IEEE Transactions on Biomedical Circuits and Systems.

[2]  Sweety Deswal,et al.  NbOx based memristor as artificial synapse emulating short term plasticity , 2019, AIP Advances.

[3]  Seung Hwan Lee,et al.  Reservoir computing using dynamic memristors for temporal information processing , 2017, Nature Communications.

[4]  Sungjun Kim,et al.  Beyond SiOx: an active electronics resurgence and biomimetic reactive oxygen species production and regulation from mitochondria , 2018 .

[5]  J. Yang,et al.  Silicon Oxide (SiOx): A Promising Material for Resistance Switching? , 2018, Advanced materials.

[6]  Kyoung-Rok Cho,et al.  Formulation and Implementation of Nonlinear Integral Equations to Model Neural Dynamics Within the Vertebrate Retina , 2018, Int. J. Neural Syst..

[7]  Wei Lu,et al.  The future of electronics based on memristive systems , 2018, Nature Electronics.

[8]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[9]  Carver A. Mead,et al.  Neuromorphic electronic systems , 1990, Proc. IEEE.

[10]  Rahimi AzghadiMostafa,et al.  2013 Special Issue , 2013 .

[11]  Kun Yue,et al.  A brain-plausible neuromorphic on-the-fly learning system implemented with magnetic domain wall analog memristors , 2019, Science Advances.

[12]  Wing H. Ng,et al.  Simulation of Inference Accuracy Using Realistic RRAM Devices , 2019, Front. Neurosci..

[13]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Sungjun Kim,et al.  Internal filament modulation in low-dielectric gap design for built-in selector-less resistive switching memory application , 2018 .

[15]  Steve B. Furber,et al.  The SpiNNaker Project , 2014, Proceedings of the IEEE.

[16]  Seung Hwan Lee,et al.  Temporal data classification and forecasting using a memristor-based reservoir computing system , 2019, Nature Electronics.

[17]  Tara Julia Hamilton,et al.  Efficient FPGA Implementations of Pair and Triplet-Based STDP for Neuromorphic Architectures , 2019, IEEE Transactions on Circuits and Systems I: Regular Papers.

[18]  Shimeng Yu,et al.  NbOx based oscillation neuron for neuromorphic computing , 2017 .

[19]  Mostafa Rahimi Azghadi,et al.  Spike-Based Synaptic Plasticity in Silicon: Design, Implementation, Application, and Challenges , 2014, Proceedings of the IEEE.

[20]  Yao-Feng Chang,et al.  Attaining resistive switching characteristics and selector properties by varying forming polarities in a single HfO2-based RRAM device with a vanadium electrode. , 2017, Nanoscale.

[21]  David W. Nauen,et al.  Coactivation and timing-dependent integration of synaptic potentiation and depression , 2005, Nature Neuroscience.

[22]  D. Abbott,et al.  Tunable Low Energy, Compact and High Performance Neuromorphic Circuit for Spike-Based Synaptic Plasticity , 2014, PloS one.

[23]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[24]  M. Bear,et al.  A biophysically-based neuromorphic model of spike rate- and timing-dependent plasticity , 2011, Proceedings of the National Academy of Sciences.

[25]  Peng Lin,et al.  Fully memristive neural networks for pattern classification with unsupervised learning , 2018 .

[26]  Giacomo Indiveri,et al.  A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128K synapses , 2015, Front. Neurosci..

[27]  J. Yang,et al.  Memristive crossbar arrays for brain-inspired computing , 2019, Nature Materials.

[28]  Hyunsang Hwang,et al.  Perspective: A review on memristive hardware for neuromorphic computation , 2018, Journal of Applied Physics.

[29]  Christof Koch,et al.  The role of single neurons in information processing , 2000, Nature Neuroscience.

[30]  Andrew S. Cassidy,et al.  TrueNorth: Accelerating From Zero to 64 Million Neurons in 10 Years , 2019, Computer.

[31]  Anthony J. Kenyon,et al.  Resistive switching in oxides , 2015 .

[32]  Kamran Eshraghian,et al.  Neuromorphic Vision Hybrid RRAM-CMOS Architecture , 2018, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[33]  Giacomo Indiveri,et al.  A differential memristive synapse circuit for on-line learning in neuromorphic computing systems , 2017, ArXiv.

[34]  Hongliang Shi,et al.  Stability and Repeatability of a Karst-like Hierarchical Porous Silicon Oxide-Based Memristor. , 2019, ACS applied materials & interfaces.

[35]  Li Ji,et al.  Integrated one diode-one resistor architecture in nanopillar SiOx resistive switching memory by nanosphere lithography. , 2014, Nano letters.

[36]  Sungjun Kim,et al.  Graphite-based selectorless RRAM: improvable intrinsic nonlinearity for array applications. , 2018, Nanoscale.

[37]  Adnan Mehonic,et al.  The interplay between structure and function in redox-based resistance switching. , 2019, Faraday discussions.

[38]  Nicolangelo Iannella,et al.  Signal Flow Platform for Mapping and Simulation of Vertebrate Retina for Sensor Systems , 2016, IEEE Sensors Journal.

[39]  R. Douglas,et al.  A silicon neuron , 1991, Nature.

[40]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[41]  S. Sze,et al.  Physics of Semiconductor Devices: Sze/Physics , 2006 .

[42]  A Mehonic,et al.  Intrinsic Resistance Switching in Amorphous Silicon Suboxides: The Role of Columnar Microstructure , 2017, Scientific Reports.

[43]  Mark Buckwell,et al.  Spike-Timing Dependent Plasticity in Unipolar Silicon Oxide RRAM Devices , 2018, Front. Neurosci..

[44]  Fei Zhou,et al.  Intrinsic SiOx-based unipolar resistive switching memory. I. Oxide stoichiometry effects on reversible switching and program window optimization , 2014 .

[45]  Mostafa Rahimi Azghadi,et al.  A neuromorphic VLSI design for spike timing and rate based synaptic plasticity , 2013, Neural Networks.

[46]  Yusuf Leblebici,et al.  Neuromorphic computing with multi-memristive synapses , 2017, Nature Communications.

[47]  C. Acha,et al.  Origin of multistate resistive switching in Ti/manganite/SiOx/Si heterostructures , 2017 .

[48]  J. Tour,et al.  Resistive switches and memories from silicon oxide. , 2010, Nano letters.

[49]  Majid Ahmadi,et al.  CORDIC-SNN: On-FPGA STDP Learning With Izhikevich Neurons , 2019, IEEE Transactions on Circuits and Systems I: Regular Papers.

[50]  Bing Chen,et al.  RRAM Crossbar Array With Cell Selection Device: A Device and Circuit Interaction Study , 2013, IEEE Transactions on Electron Devices.

[51]  Johannes Schemmel,et al.  Demonstrating Advantages of Neuromorphic Computation: A Pilot Study , 2018, Front. Neurosci..

[52]  Giacomo Indiveri,et al.  Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI , 2009, IEEE Transactions on Biomedical Circuits and Systems.

[53]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[54]  Kwabena Boahen,et al.  Braindrop: A Mixed-Signal Neuromorphic Architecture With a Dynamical Systems-Based Programming Model , 2019, Proceedings of the IEEE.

[55]  T. W. Hickmott.  LOW-FREQUENCY NEGATIVE RESISTANCE IN THIN ANODIC OXIDE FILMS , 1962 .

[56]  Peng Lin,et al.  Reinforcement learning with analogue memristor arrays , 2019, Nature Electronics.

[57]  Jie-Ming Wang,et al.  LiSiOX-Based Analog Memristive Synapse for Neuromorphic Computing , 2019, IEEE Electron Device Letters.

[58]  K. Usha,et al.  Hindmarsh-Rose neuron model with memristors , 2019, Biosyst..

[59]  Rui Yang,et al.  Memristive Synapses and Neurons for Bioinspired Computing , 2019, Advanced Electronic Materials.

[60]  Bertrand Fontaine,et al.  Fitting Neuron Models to Spike Trains , 2011, Front. Neurosci..

[61]  Fei Zhou,et al.  Demonstration of Synaptic Behaviors and Resistive Switching Characterizations by Proton Exchange Reactions in Silicon Oxide , 2016, Scientific Reports.

[62]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[63]  Shimeng Yu,et al.  Emerging Memory Technologies: Recent Trends and Prospects , 2016, IEEE Solid-State Circuits Magazine.

[64]  L. Chua.  Memristor, Hodgkin–Huxley, and Edge of Chaos , 2013, Nanotechnology.

[65]  Giacomo Indiveri,et al.  A neuromorphic systems approach to in-memory computing with non-ideal memristive devices: From mitigation to exploitation , 2018, Faraday discussions.

[66]  M. Pickett,et al.  A scalable neuristor built with Mott memristors. , 2013, Nature materials.

[67]  Bernabe Linares-Barranco,et al.  A Hybrid CMOS-Memristor Neuromorphic Synapse , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[68]  J. Simmons,et al.  New conduction and reversible memory phenomena in thin insulating films , 1967, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[69]  M. Bear,et al.  Experience-dependent modification of synaptic plasticity in visual cortex , 1996, Nature.

[70]  Byung-Gook Park,et al.  Analog Synaptic Behavior of a Silicon Nitride Memristor. , 2017, ACS applied materials & interfaces.

[71]  Wei Yi,et al.  Biological plausibility and stochasticity in scalable VO2 active memristor neurons , 2018, Nature Communications.

[72]  Bike Xie,et al.  Integrating memristors and CMOS for better AI , 2019, Nature Electronics.

[73]  Fei Zhou,et al.  Study of self-compliance behaviors and internal filament characteristics in intrinsic SiOx-based resistive switching memory , 2016 .

[74]  Giacomo Indiveri,et al.  A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity , 2006, IEEE Transactions on Neural Networks.

[75]  C. Morris,et al.  Voltage oscillations in the barnacle giant muscle fiber. , 1981, Biophysical journal.

[76]  Miao Hu,et al.  ISAAC: A Convolutional Neural Network Accelerator with In-Situ Analog Arithmetic in Crossbars , 2016, 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA).

[77]  James M Tour,et al.  Nanoporous silicon oxide memory. , 2014, Nano letters.

[78]  Heiner Giefers,et al.  Mixed-precision in-memory computing , 2017, Nature Electronics.

[79]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[80]  Aldenor G. Santos,et al.  Occurrence of the potent mutagens 2- nitrobenzanthrone and 3-nitrobenzanthrone in fine airborne particles , 2019, Scientific Reports.

[81]  Ru Huang,et al.  Engineering incremental resistive switching in TaOx based memristors for brain-inspired computing. , 2016, Nanoscale.

[82]  Jue Xiong,et al.  Quasi‐Hodgkin–Huxley Neurons with Leaky Integrate‐and‐Fire Functions Physically Realized with Memristive Devices , 2018, Advanced materials.

[83]  Runchen Fang,et al.  A CMOS-compatible electronic synapse device based on Cu/SiO2/W programmable metallization cells , 2016, Nanotechnology.

[84]  Zhengya Zhang,et al.  A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operations , 2019, Nature Electronics.

[85]  Wei Yang Lu,et al.  Nanoscale memristor device as synapse in neuromorphic systems. , 2010, Nano letters.

[86]  Anthony J. Kenyon,et al.  Intrinsic resistance switching in amorphous silicon oxide for high performance SiOx ReRAM devices , 2017 .

[87]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[88]  Adnan Mehonic,et al.  Emulating the Electrical Activity of the Neuron Using a Silicon Oxide RRAM Cell , 2016, Front. Neurosci..

[89]  James B. Aimone,et al.  Memristors learn to play , 2019 .

[90]  巩岩 Gong Yan,et al.  Vector Analysis of Diffractive Optical Elements for Off-Axis Illumination of Projection Lithographic System , 2011 .

[91]  Mostafa Rahimi Azghadi,et al.  Programmable Spike-Timing-Dependent Plasticity Learning Circuits in Neuromorphic VLSI Architectures , 2015, ACM J. Emerg. Technol. Comput. Syst..

[92]  H.-S. Philip Wong,et al.  In-memory computing with resistive switching devices , 2018, Nature Electronics.

[93]  Leon O. Chua,et al.  Neuromemristive Circuits for Edge Computing: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.