Self-Attention ConvLSTM and Its Application in RUL Prediction of Rolling Bearings

Traditional long short-term memory (LSTM) neural networks generally face the challenge of low training efficiency and poor prediction accuracy for the remaining useful life (RUL) prediction due to their structure. In this study, a novel model called self-attention ConvLSTM (SA-ConvLSTM) neural network is proposed derived from ConvLSTM and a SA mechanism. First, convolution operators replace the fully connected layers inside the network structure to reduce the redundancy of the network and enhance its nonlinear modeling capability. Subsequently, a SA module is designed and embedded into the interior of the model by adaptively employing the corresponding important information to improve the prediction performance. Extensive experiments on the test rig and the actual wind farm confirmed that the developed SA-ConvLSTM has advantages over other conventional prediction methods in terms of convergence speed and prediction precision.

[1]  Zepeng Liu,et al.  A review of failure modes, condition monitoring and fault diagnosis methods for large-scale wind turbine bearings , 2020 .

[2]  Yi Qin,et al.  Gated Dual Attention Unit Neural Networks for Remaining Useful Life Prediction of Rolling Bearings , 2021, IEEE Transactions on Industrial Informatics.

[3]  Aedín C. Culhane,et al.  Dimension reduction techniques for the integrative analysis of multi-omics data , 2016, Briefings Bioinform..

[4]  Dit-Yan Yeung,et al.  Spatiotemporal Modeling for Crowd Counting in Videos , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[5]  Brigitte Chebel-Morello,et al.  PRONOSTIA : An experimental platform for bearings accelerated degradation tests. , 2012 .

[6]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Yaguo Lei,et al.  Deep separable convolutional network for remaining useful life prediction of machinery , 2019 .

[8]  Liang Guo,et al.  Machinery health indicator construction based on convolutional neural networks considering trend burr , 2018, Neurocomputing.

[9]  Sanyuan Zhao,et al.  Pyramid Dilated Deeper ConvLSTM for Video Salient Object Detection , 2018, ECCV.

[10]  Yaguo Lei,et al.  Machinery health prognostics: A systematic review from data acquisition to RUL prediction , 2018 .

[11]  Tzu-Liang Tseng,et al.  Degradation modeling and RUL prediction using Wiener process subject to multiple change points and unit heterogeneity , 2018, Reliab. Eng. Syst. Saf..

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  Garrison W. Cottrell,et al.  A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction , 2017, IJCAI.

[14]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[15]  Zhenghua Chen,et al.  Machine Remaining Useful Life Prediction via an Attention-Based Deep Learning Approach , 2021, IEEE Transactions on Industrial Electronics.

[16]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[17]  Qian Liu,et al.  Gated recurrent unit based recurrent neural network for remaining useful life prediction of nonlinear deterioration process , 2019, Reliab. Eng. Syst. Saf..

[18]  Yi Wang,et al.  Rolling Bearing Fault Detection of Civil Aircraft Engine Based on Adaptive Estimation of Instantaneous Angular Speed , 2020, IEEE Transactions on Industrial Informatics.

[19]  Peng Wang,et al.  Long short-term memory for machine remaining life prediction , 2018, Journal of Manufacturing Systems.

[20]  Liang Guo,et al.  A recurrent neural network based health indicator for remaining useful life prediction of bearings , 2017, Neurocomputing.

[21]  Yi Zhang,et al.  Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? , 2020, ICLR.

[22]  Hongxun Yao,et al.  Auto-encoder based dimensionality reduction , 2016, Neurocomputing.

[23]  Enrico Zio,et al.  Remaining Useful Life Prediction Based on a Double-Convolutional Neural Network Architecture , 2019, IEEE Transactions on Industrial Electronics.

[24]  Muhammad Zubair Asghar,et al.  Exploring deep neural networks for rumor detection , 2019, Journal of Ambient Intelligence and Humanized Computing.

[25]  Yaguo Lei,et al.  Condition monitoring and fault diagnosis of planetary gearboxes: A review , 2014 .

[26]  Yangyang Wang,et al.  Long short-term memory neural network with weight amplification and its application into gear remaining useful life prediction , 2020, Eng. Appl. Artif. Intell..

[27]  W. Y. Liu,et al.  The structure healthy condition monitoring and fault diagnosis methods in wind turbines: A review , 2015 .

[28]  Haizhou Chen,et al.  Long-term gear life prediction based on ordered neurons LSTM neural networks , 2020 .

[29]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[30]  Fei Teng,et al.  A Deep Learning-Based Remaining Useful Life Prediction Approach for Bearings , 2020, IEEE/ASME Transactions on Mechatronics.

[31]  Qinkai Han,et al.  Vibration based condition monitoring and fault diagnosis of wind turbine planetary gearbox: A review , 2019, Mechanical Systems and Signal Processing.

[32]  Zhiyu Zhu,et al.  A novel deep learning method based on attention mechanism for bearing remaining useful life prediction , 2020, Appl. Soft Comput..

[33]  Jie Chen,et al.  Performance degradation assessment of a wind turbine gearbox based on multi-sensor data fusion , 2019, Mechanism and Machine Theory.

[34]  Lei Deng,et al.  Condition monitoring of wind turbines based on spatio-temporal fusion of SCADA data by convolutional neural networks and gated recurrent units , 2020 .

[35]  Yi Qin,et al.  LSTM networks based on attention ordered neurons for gear remaining life prediction. , 2020, ISA transactions.

[36]  Klaus Dietmayer,et al.  Separable Convolutional LSTMs for Faster Video Segmentation , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[37]  Weiwen Peng,et al.  Estimation of Bearing Remaining Useful Life Based on Multiscale Convolutional Neural Network , 2019, IEEE Transactions on Industrial Electronics.

[38]  Guoqiang Zhong,et al.  Recurrent Attention Unit , 2018, ArXiv.

[39]  Bin Zhang,et al.  Degradation Feature Selection for Remaining Useful Life Prediction of Rolling Element Bearings , 2016, Qual. Reliab. Eng. Int..

[40]  Hao Zhang,et al.  Attention-Based LSTM Network for Rotatory Machine Remaining Useful Life Prediction , 2020, IEEE Access.

[41]  Jian Cheng,et al.  Life Prediction for Machinery Components Based on CNN-BiLSTM Network and Attention Model , 2020, 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC).