Circuit Design of Memristor-based GRU and its Applications in SOC Estimation

Gated recurrent unit (GRU) is a variant of recurrent neural network (RNN), which is widely used in applications and tasks related to sequence data processing. However, traditional GRU networks based on von Neumann computing construction, have been facing challenges such as big data dimension and high real-time requirements. Meanwhile, limited internal storage resources and external storage bandwidth have jointly limited the overall performance of hardware implementation of GRU networks. Based on these, a compact scheme for the hardware circuit design of memristor-based GRU network is presented, along with the concrete circuit design of nonlinear activation and the linear matrix operation. The entire scheme is validated by the application of the Lithium ion battery state of charge (SOC) estimation. This work is expected to integrate the neuromorphic electronics and battery management systems, and further promote the development of electric vehicles in smart cities.

[1]  Donglian Qi,et al.  Multimodal Neuromorphic Sensory-Processing System With Memristor Circuits for Smart Home Applications , 2023, IEEE Transactions on Industry Applications.

[2]  Jianhui Zhao,et al.  A review of Mott insulator in memristors: The materials, characteristics, applications for future computing systems and neuromorphic computing , 2022, Nano Research.

[3]  Yuchao Yang,et al.  Dynamical memristors for higher-complexity neuromorphic computing , 2022, Nature Reviews Materials.

[4]  M. Woźniak,et al.  Recurrent neural network model for high-speed train vibration prediction from time series , 2022, Neural Computing and Applications.

[5]  Naeem Seliya,et al.  Applications of Recurrent Neural Network for Biometric Authentication & Anomaly Detection , 2021, Inf..

[6]  Zaghloul Saad Zaghloul,et al.  The FPGA Hardware Implementation of the Gated Recurrent Unit Architecture , 2021, SoutheastCon 2021.

[7]  Dalibor Biolek,et al.  (V)TEAM for SPICE Simulation of Memristive Devices With Improved Numerical Performance , 2021, IEEE Access.

[8]  Ashok Sarabu,et al.  Human Action Recognition in Videos using Convolution Long Short-Term Memory Network with Spatio-Temporal Networks , 2021 .

[9]  Zhiqiang Que,et al.  Optimizing Reconfigurable Recurrent Neural Networks , 2020, 2020 IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM).

[10]  Muhammad Khalid.  Review on Various Memristor Models, Characteristics, Potential Applications, and Future Works , 2019, Transactions on Electrical and Electronic Materials.

[11]  Nanning Zheng,et al.  Design Space Exploration of Neural Network Activation Function Circuits , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[12]  Yongxin Zhu,et al.  Improving Prediction Accuracy in LSTM Network Model for Aircraft Testing Flight Data , 2018, 2018 IEEE International Conference on Smart Cloud (SmartCloud).

[13]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[14]  Shuai Li,et al.  Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Houshang Darabi,et al.  Multivariate LSTM-FCNs for Time Series Classification , 2018, Neural Networks.

[16]  Jürgen Ziegler,et al.  Sequential User-based Recurrent Neural Network Recommendations , 2017, RecSys.

[17]  Eriko Nurvitadhi,et al.  Accelerating recurrent neural networks in analytics servers: Comparison of FPGA, CPU, GPU, and ASIC , 2016, 2016 26th International Conference on Field Programmable Logic and Applications (FPL).

[18]  Steven D. Lacey,et al.  Organic electrode for non-aqueous potassium-ion batteries , 2015 .

[19]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[20]  Narayan Srinivasa,et al.  A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. , 2012, Nano letters.

[21]  D. Stewart,et al.  The missing memristor found , 2008, Nature.

[22]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[23]  Zhiwei He,et al.  An Improved Bidirectional Gated Recurrent Unit Method for Accurate State-of-Charge Estimation , 2021, IEEE Access.

[24]  Ali Shariq Imran,et al.  Human action recognition using attention based LSTM network with dilated CNN features , 2021, Future Gener. Comput. Syst..

[25]  Donglian Qi,et al.  TSSM: Three-State Switchable Memristor Model Based on Ag/TiOx Nanobelt/Ti Configuration , 2021, Int. J. Bifurc. Chaos.

[26]  C. Mahata,et al.  Demonstration of synaptic and resistive switching characteristics in W/TiO2/HfO2/TaN memristor crossbar array for bioinspired neuromorphic computing , 2022 .