Reliability of Memristive Devices for High-Performance Neuromorphic Computing: (Invited Paper)

With the rich internal ion dynamics, memristor-based neuromorphic computing emerges as a non-von Neumann computing paradigm to mimic biological neural networks and achieve high energy efficiency. However, to implement large-scale memristive neural networks, the reliability issue of memristive devices, including artificial synapse, dendrite, and soma, should be properly addressed. In this paper, recent works investigating the physical mechanisms and optimizations of memristive device reliability are presented. In particular, the relaxation effect of $\boldsymbol{\text{HfO}_{\mathrm{x}}}$ -based artificial synapse is alleviated by using a ternary oxide as the thermal enhance layer, the device yield of $\boldsymbol{\text{TiO}_{\mathrm{x}^{-}}}$ based artificial dendrite is improved by proper material selection and interface engineering, and the device variability of $\boldsymbol{\text{NbO}_{\mathrm{x}}}$ -based artificial soma is reduced by nitrogen doping. Furthermore, a bio-inspired dendritic neural network with these three fundamental memristive devices is constructed and simulated to analyze the influence of device reliability. Using these optimized devices, the classification accuracy of the street-view house number dataset can be improved by up to $\sim$ 60%. The quantitative requirements of device reliability metrics are also provided as a guideline for future neuromorphic system design and implementation.

[1]  Huaqiang Wu,et al.  A Hybrid Computing-In-Memory Architecture by Monolithic 3D Integration of BEOL CNT/IGZO-based CFET Logic and Analog RRAM , 2022, International Electron Devices Meeting.

[2]  Huaqiang Wu,et al.  Microscopic Modeling and Optimization of NbOx Mott Memristor for Artificial Neuron Applications , 2022, IEEE Transactions on Electron Devices.

[3]  Huaqiang Wu,et al.  A memristor-based analogue reservoir computing system for real-time and power-efficient signal processing , 2022, Nature Electronics.

[4]  Huaqiang Wu,et al.  The Impact of Thermal Enhance Layers on the Relaxation Effect in Analog RRAM , 2022, IEEE Transactions on Electron Devices.

[5]  Huaqiang Wu,et al.  Memristors-based Dendritic Neuron for High-Efficiency Spatial-Temporal Information Processing. , 2022, Advanced materials.

[6]  Joon-Kyu Han,et al.  A Review of Artificial Spiking Neuron Devices for Neural Processing and Sensing , 2022, Advanced Functional Materials.

[7]  Shinhyun Choi,et al.  Experimental demonstration of highly reliable dynamic memristor for artificial neuron and neuromorphic computing , 2022, Nature Communications.

[8]  Huaqiang Wu,et al.  Rotating neurons for all-analog implementation of cyclic reservoir computing , 2022, Nature Communications.

[9]  Huaqiang Wu,et al.  Investigation of Resistive Switching Mechanisms in Ti/TiOx/Pd‐Based RRAM Devices , 2021, Advanced Electronic Materials.

[10]  Huaqiang Wu,et al.  Monolithic 3D Integration of Logic, Memory and Computing-In-Memory for One-Shot Learning , 2021, 2021 IEEE International Electron Devices Meeting (IEDM).

[11]  Lifeng Liu,et al.  Improvement of State Stability in Multi-Level Resistive Random-Access Memory (RRAM) Array for Neuromorphic Computing , 2021, IEEE Electron Device Letters.

[12]  Huaqiang Wu,et al.  Oscillation neuron based on a low-variability threshold switching device for high-performance neuromorphic computing , 2021 .

[13]  Meng-Fan Chang,et al.  In-memory Learning with Analog Resistive Switching Memory: A Review and Perspective , 2021, Proceedings of the IEEE.

[14]  Ming Liu,et al.  Fully Memristive SNNs with Temporal Coding for Fast and Low-power Edge Computing , 2020, 2020 IEEE International Electron Devices Meeting (IEDM).

[15]  Javier Del Ser,et al.  Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions , 2020, IEEE Transactions on Intelligent Transportation Systems.

[16]  Huaqiang Wu,et al.  High‐Uniformity Threshold Switching HfO2‐Based Selectors with Patterned Ag Nanodots , 2020, Advanced science.

[17]  Wei Wu,et al.  Power-efficient neural network with artificial dendrites , 2020, Nature Nanotechnology.

[18]  Huaqiang Wu,et al.  Impact and Quantization of Short-Term Relaxation effect in Analog RRAM , 2020, 2020 4th IEEE Electron Devices Technology & Manufacturing Conference (EDTM).

[19]  Meng-Fan Chang,et al.  33.2 A Fully Integrated Analog ReRAM Based 78.4TOPS/W Compute-In-Memory Chip with Fully Parallel MAC Computing , 2020, 2020 IEEE International Solid- State Circuits Conference - (ISSCC).

[20]  Bin Gao,et al.  Fully hardware-implemented memristor convolutional neural network , 2020, Nature.

[21]  Sen Song,et al.  Bridging Biological and Artificial Neural Networks with Emerging Neuromorphic Devices: Fundamentals, Progress, and Challenges , 2019, Advanced materials.

[22]  Qi Liu,et al.  Nb1-xO2 based Universal Selector with Ultra-high Endurance (>1012), high speed (10ns) and Excellent Vth Stability , 2019, 2019 Symposium on VLSI Technology.

[23]  Hyunsang Hwang,et al.  Various Threshold Switching Devices for Integrate and Fire Neuron Applications , 2019, Advanced Electronic Materials.

[24]  T. Bieler,et al.  An analysis of (the lack of) slip transfer between near-cube oriented grains in pure Al , 2019, International Journal of Plasticity.

[25]  Qing Luo,et al.  40× Retention Improvement by Eliminating Resistance Relaxation with High Temperature Forming in 28 nm RRAM Chip , 2018, 2018 IEEE International Electron Devices Meeting (IEDM).

[26]  Xiaoyu Sun,et al.  Characterizing Endurance Degradation of Incremental Switching in Analog RRAM for Neuromorphic Systems , 2018, 2018 IEEE International Electron Devices Meeting (IEDM).

[27]  Shimeng Yu,et al.  Demonstration of Generative Adversarial Network by Intrinsic Random Noises of Analog RRAM Devices , 2018, 2018 IEEE International Electron Devices Meeting (IEDM).

[28]  J. Yang,et al.  Efficient and self-adaptive in-situ learning in multilayer memristor neural networks , 2018, Nature Communications.

[29]  H. Hwang,et al.  Hybrid Selector With Excellent Selectivity and Fast Switching Speed for X-Point Memory Array , 2018, IEEE Electron Device Letters.

[30]  J. Cong,et al.  Scaling for edge inference of deep neural networks , 2018 .

[31]  Subhrajit Roy,et al.  Spiking Neural Classifier with Lumped Dendritic Nonlinearity and Binary Synapses: A Current Mode VLSI Implementation and Analysis , 2018, Neural Computation.

[32]  Shimeng Yu,et al.  Investigation of statistical retention of filamentary analog RRAM for neuromophic computing , 2017, 2017 IEEE International Electron Devices Meeting (IEDM).

[33]  Shimeng Yu,et al.  NbOx based oscillation neuron for neuromorphic computing , 2017 .

[34]  Shimeng Yu,et al.  Improving Analog Switching in HfOx-Based Resistive Memory With a Thermal Enhanced Layer , 2017, IEEE Electron Device Letters.

[35]  David A. Patterson,et al.  In-datacenter performance analysis of a tensor processing unit , 2017, 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA).

[36]  Johannes Schemmel,et al.  An accelerated analog neuromorphic hardware system emulating NMDA- and calcium-based non-linear dendrites , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[37]  L. Goux,et al.  Intrinsic Tailing of Resistive States Distributions in Amorphous HfOx and TaOx Based Resistive Random Access Memories , 2015, IEEE Electron Device Letters.

[38]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[39]  J. Yang,et al.  High switching endurance in TaOx memristive devices , 2010 .

[40]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[41]  Kikuo Yamabe,et al.  First-principles studies of the intrinsic effect of nitrogen atoms on reduction in gate leakage current through Hf-based high-k dielectrics , 2005 .

[42]  Giacomo Indiveri,et al.  A low-power adaptive integrate-and-fire neuron circuit , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[43]  A. Cabrini,et al.  Intrinsic program instability in HfO2 RRAM and consequences on program algorithms , 2015, 2015 IEEE International Electron Devices Meeting (IEDM).

[44]  MD James H. Thrall,et al.  Arti fi cial Intelligence and Machine Learning in Radiology: Opportunities, Challenges, Pitfalls, and Criteria for Success , 2022 .