Reduction 93.7% time and power consumption using a memristor-based imprecise gradient update algorithm

The conventional computing system with the architecture of von Neumann has greatly benefited our humans for past decades, while it is also suffered from low efficiency due to the separation between a memory block and a processing unit. Memristor, which is an emerging electron device with the capability of data storage and processing information simultaneously, can be employed to construct a bioinspired neuromorphic computing system. Simulation as one of the most powerful methods to obtain the optimizing result for the memristor-based neuromorphic network has been extensively focused to realize the high precision calculation. It becomes very difficult because the pulse-to-pulse (P2P) model is limited by the updating process. The memristor-based multi-layer Perceptron (MLP) network online training generally presents a low accuracy. Therefore, an efficiency training schedual is urgently desired to improve the accuracy. Based on the resistive switching behavior observed in the Ag/TiOx/F-doped SnO 2 memristor, the weight update by the P2P model enables the MLP network online training in the low accuracy memristor with high performance. The low bits MLP optimized by a novel weight update schedual can realize high precision identification and classification. By that, the time and power consumption of memristor can be largely reduced. The experiment result illustrates that the high accuracy of 90.82% and 95.44% can be obtained at the first and final epoch of the MNIST handwritten digital datasets, respectively. Importantly, the number of the weight update, and the online training time and power consumption can be reduced by 81% and 93.7%, respectively. The scheme provides high precision, low power consumption, and fast convergence solution for the in-situ training of the imprecise memristor-based neuromorphic network.

[1]  Wenqiang Zhang,et al.  Sign backpropagation: An on-chip learning algorithm for analog RRAM neuromorphic computing systems , 2018, Neural Networks.

[2]  Shukai Duan,et al.  Artificial and wearable albumen protein memristor arrays with integrated memory logic gate functionality , 2019, Materials Horizons.

[3]  Yu-Rong Zeng,et al.  Effective sparse adaboost method with ESN and FOA for industrial electricity consumption forecasting in China , 2018, Energy.

[4]  Wei-Ho Tsai,et al.  Improving search engine optimization (SEO) by using hybrid modified MCDM models , 2018, Artificial Intelligence Review.

[5]  Ning Xu,et al.  A review for cervical histopathology image analysis using machine vision approaches , 2020, Artificial Intelligence Review.

[6]  Penghong Wang,et al.  Weight convergence analysis of DV-hop localization algorithm with GA , 2020, Soft Comput..

[7]  Shukai Duan,et al.  Negative Photoconductance Effect: An Extension Function of the TiOx‐Based Memristor , 2021, Advanced Science.

[8]  Qing Zhu,et al.  Effective long short-term memory with fruit fly optimization algorithm for time series forecasting , 2020, Soft Computing.

[9]  Ligang Gao,et al.  Programming Protocol Optimization for Analog Weight Tuning in Resistive Memories , 2015, IEEE Electron Device Letters.

[10]  Pritish Narayanan,et al.  Equivalent-accuracy accelerated neural-network training using analogue memory , 2018, Nature.

[11]  Ying Zhu,et al.  Image segmentation evaluation: a survey of methods , 2020, Artificial Intelligence Review.

[12]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[13]  Lu Peng,et al.  Effective energy consumption forecasting using enhanced bagged echo state network , 2020 .

[14]  Shukai Duan,et al.  Resistive switching memory integrated with amorphous carbon-based nanogenerators for self-powered device , 2019, Nano Energy.

[15]  Peng Lin,et al.  Reinforcement learning with analogue memristor arrays , 2019, Nature Electronics.

[16]  Wei D. Lu,et al.  Sparse coding with memristor networks. , 2017, Nature nanotechnology.

[17]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[18]  Lei Tian,et al.  SSM: a high-performance scheme for in situ training of imprecise memristor neural networks , 2020, Neurocomputing.

[19]  Bin Gao,et al.  Fully hardware-implemented memristor convolutional neural network , 2020, Nature.

[20]  Guangdong Zhou,et al.  A Unified Capacitive-Coupled Memristive Model for the Non-Pinched Current-Voltage Hysteresis Loop. , 2019, Nano letters.

[21]  Asifullah Khan,et al.  A survey of the recent architectures of deep convolutional neural networks , 2019, Artificial Intelligence Review.

[22]  A. Bhardwaj,et al.  In situ click chemistry generation of cyclooxygenase-2 inhibitors , 2017, Nature Communications.

[23]  Shukai Duan,et al.  Coexistence of Negative Differential Resistance and Resistive Switching Memory at Room Temperature in TiOx Modulated by Moisture , 2018 .

[24]  Kongming Wu,et al.  Construction and analysis of cDNA libraries from the antennae of male and female cotton bollworms Helicoverpa armigera (Hübner) and expression analysis of putative odorant-binding protein genes. , 2011, Biochemical and biophysical research communications.

[25]  O. Krestinskaya,et al.  Memristive GAN in Analog , 2020, Scientific Reports.

[26]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[27]  Hongwei Yao,et al.  Corrigendum: Adenylate kinase hCINAP determines self-renewal of colorectal cancer stem cells by facilitating LDHA phosphorylation , 2017, Nature Communications.

[28]  Bo Hong,et al.  Neural signal analysis with memristor arrays towards high-efficiency brain–machine interfaces , 2020, Nature Communications.

[29]  Penousal Machado,et al.  Image descriptors in radiology images: a systematic review , 2017, Artificial Intelligence Review.

[30]  Gregory S. Snider,et al.  ‘Memristive’ switches enable ‘stateful’ logic operations via material implication , 2010, Nature.

[31]  M. Mitchell Waldrop,et al.  The chips are down for Moore’s law , 2016, Nature.

[32]  Shukai Duan,et al.  Capacitive effect: An original of the resistive switching memory , 2020 .

[33]  Oscar Castillo,et al.  Special issue on “Extensions to type-1 fuzzy logic: theory, algorithms and applications” , 2020, Soft Comput..

[34]  William H. Kautz,et al.  Cellular Logic-in-Memory Arrays , 1969, IEEE Transactions on Computers.

[35]  L. Chua Memristor-The missing circuit element , 1971 .

[36]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[37]  Farnood Merrikh-Bayat,et al.  Training and operation of an integrated neuromorphic network based on metal-oxide memristors , 2014, Nature.

[38]  H.-S. Philip Wong,et al.  Face classification using electronic synapses , 2017, Nature Communications.