An ensemble framework based on convolutional bi-directional LSTM with multiple time windows for remaining useful life estimation

Abstract Effectively estimating remaining useful life (RUL) is crucially important for evaluating machine health. In the industry, there exists a high degree of inconsistency among the length of condition monitoring data. Thus, we propose an ensemble framework based on convolutional bi-directional long short-term memory with multiple time windows (MTW CNN-BLSTM Ensemble) for accurately predicting RUL under this circumstance. In the training phase, multiple CNN-BLSTM base models with different time window sizes are trained to capture various temporal dependencies between features. This setting expands the time window size and reduces the training error compared to traditional static time window size approaches. In the testing phase, test units are classified and suitable base models are applied according to the length of running time. A weighted average method is exploited to aggregate base models’ outcomes. This ensemble strategy can increase the utilization rate of the test data and further enhance prediction accuracy. The effectiveness of this framework is validated and the comparison with state-of-the-art methods available has been provided. The results have shown that this framework can achieve the minimum prediction error and provide stable support for equipment health management.

[1]  Uzay Kaymak,et al.  Remaining Useful Lifetime Prediction via Deep Domain Adaptation , 2019, Reliab. Eng. Syst. Saf..

[2]  Jianbo Yu,et al.  A selective deep stacked denoising autoencoders ensemble with negative correlation learning for gearbox fault diagnosis , 2019, Comput. Ind..

[3]  Sanghoon Lee,et al.  Ensemble Deep Learning for Skeleton-Based Action Recognition Using Temporal Sliding LSTM Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Jehn-Ruey Jiang,et al.  Remaining useful life estimation using long short-term memory deep learning , 2018, 2018 IEEE International Conference on Applied System Invention (ICASI).

[5]  Guilin Wen,et al.  Remaining Useful Life Estimation in Prognostics Using Deep Bidirectional LSTM Neural Network , 2018, 2018 Prognostics and System Health Management Conference (PHM-Chongqing).

[6]  João Gama,et al.  Ensemble learning for data stream analysis: A survey , 2017, Inf. Fusion.

[7]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[8]  Zhigang Tian,et al.  Uncertainty Quantification in Gear Remaining Useful Life Prediction Through an Integrated Prognostics Method , 2013, IEEE Transactions on Reliability.

[9]  Amir Asif,et al.  A multimodal and hybrid deep neural network model for Remaining Useful Life estimation , 2019, Comput. Ind..

[10]  Gavin Brown,et al.  Diversity and degrees of freedom in regression ensembles , 2018, Neurocomputing.

[11]  Guangzhong Dong,et al.  Remaining Useful Life Prediction and State of Health Diagnosis for Lithium-Ion Batteries Using Particle Filter and Support Vector Regression , 2018, IEEE Transactions on Industrial Electronics.

[12]  Juan Antonio Álvarez,et al.  Deep neural network for traffic sign recognition systems: An analysis of spatial transformers and stochastic optimisation methods , 2018, Neural Networks.

[13]  Tangbin Xia,et al.  Recent advances in prognostics and health management for advanced manufacturing paradigms , 2018, Reliab. Eng. Syst. Saf..

[14]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Chetan Gupta,et al.  Long Short-Term Memory Network for Remaining Useful Life estimation , 2017, 2017 IEEE International Conference on Prognostics and Health Management (ICPHM).

[16]  Peng Wang,et al.  Long short-term memory for machine remaining life prediction , 2018, Journal of Manufacturing Systems.

[17]  Soumaya Yacout,et al.  Bidirectional handshaking LSTM for remaining useful life prediction , 2019, Neurocomputing.

[18]  Hayaru Shouno,et al.  Analysis of Dropout Learning Regarded as Ensemble Learning , 2016, ICANN.

[19]  Navdeep Jaitly,et al.  Hybrid speech recognition with Deep Bidirectional LSTM , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[20]  Enrico Zio,et al.  Echo State Network for the Remaining Useful Life Prediction of a Turbofan Engine , 2016, PHM Society European Conference.

[21]  F.O. Heimes,et al.  Recurrent neural networks for remaining useful life estimation , 2008, 2008 International Conference on Prognostics and Health Management.

[22]  Chao Hu,et al.  Ensemble of data-driven prognostic algorithms for robust prediction of remaining useful life , 2011, 2011 IEEE Conference on Prognostics and Health Management.

[23]  Ali Ouni,et al.  Optimal Deep Learning LSTM Model for Electric Load Forecasting using Feature Selection and Genetic Algorithm: Comparison with Machine Learning Approaches † , 2018, Energies.

[24]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[25]  Sandeep Kumar,et al.  A novel soft computing method for engine RUL prediction , 2017, Multimedia Tools and Applications.

[26]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[27]  Kay Chen Tan,et al.  A time window neural network based framework for Remaining Useful Life estimation , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[28]  Dong Dong,et al.  Life prediction of jet engines based on LSTM-recurrent neural networks , 2017, 2017 Prognostics and System Health Management Conference (PHM-Harbin).

[29]  Enrico Zio,et al.  Ensemble of optimized echo state networks for remaining useful life prediction , 2017, Neurocomputing.

[30]  Weiwen Peng,et al.  Estimation of Bearing Remaining Useful Life Based on Multiscale Convolutional Neural Network , 2019, IEEE Transactions on Industrial Electronics.

[31]  Li Lin,et al.  Remaining useful life estimation of engineered systems using vanilla LSTM neural networks , 2018, Neurocomputing.

[32]  P. J. García Nieto,et al.  Hybrid PSO-SVM-based method for forecasting of the remaining useful life for aircraft engines and evaluation of its reliability , 2015, Reliab. Eng. Syst. Saf..

[33]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Xiaoli Li,et al.  Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life , 2016, DASFAA.

[35]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[36]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[37]  Kwok L. Tsui,et al.  A naive Bayes model for robust remaining useful life prediction of lithium-ion battery , 2014 .

[38]  Hong Jiang,et al.  A novel Switching Unscented Kalman Filter method for remaining useful life prediction of rolling bearing , 2019, Measurement.

[39]  Noureddine Zerhouni,et al.  Degradations analysis and aging modeling for health assessment and prognostics of PEMFC , 2016, Reliab. Eng. Syst. Saf..

[40]  Heiko Paulheim,et al.  Ensembles of Recurrent Neural Networks for Robust Time Series Forecasting , 2017, SGAI Conf..

[41]  Noureddine Zerhouni,et al.  State of the art and taxonomy of prognostics approaches, trends of prognostics applications and open issues towards maturity at different technology readiness levels , 2017 .

[42]  David He,et al.  Using Deep Learning-Based Approach to Predict Remaining Useful Life of Rotating Components , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[43]  Kai Goebel,et al.  A neural network filtering approach for similarity-based remaining useful life estimation , 2018, The International Journal of Advanced Manufacturing Technology.

[44]  C. Kandler,et al.  A new framework for remaining useful life estimation using Support Vector Machine classifier , 2013, 2013 Conference on Control and Fault-Tolerant Systems (SysTol).

[45]  Shuo Li,et al.  A Data-driven Approach for Remaining Useful Life Prediction of Aircraft Engines , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[46]  Xiaofeng Hu,et al.  Remaining useful life prediction based on health index similarity , 2019, Reliab. Eng. Syst. Saf..

[47]  Bin Zhang,et al.  Bearing performance degradation assessment using long short-term memory recurrent network , 2019, Comput. Ind..

[48]  Ruqiang Yan,et al.  Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks , 2017, Sensors.

[49]  Yaguo Lei,et al.  Degradation data analysis and remaining useful life estimation: A review on Wiener-process-based methods , 2018, Eur. J. Oper. Res..