A Joint Long Short-Term Memory and AdaBoost regression approach with application to remaining useful life estimation

Abstract Along with wide application of sensors, multi-dimensional time-series data are commonly available for remaining useful life (RUL) estimation. This paper proposes a joint data-driven approach that adapts two models, AdaBoost regression and Long Short-Term Memory (LSTM), to estimate the RUL based on data trajectory extension. In RUL prediction, the data trajectories in the training set contain the data up to the units’ failure while the data trajectories in the testing set do not. Although this fact has a significant negative effect on the accuracy of RUL estimation, it is considered by few literatures. The proposed approach adapts the LSTM to learn the time series dependencies of training data and then extend the trajectories of testing data, aiming at reducing the variance of the lengths of data trajectory between the training and testing sets. Then, the proposed approach adapts the AdaBoost regression to estimate the RUL using the extended time series data. The proposed approach is competitive with state-of-the-art methods by demonstrating on two degradation datasets.

[1]  Ming Chen,et al.  A data-driven model for milling tool remaining useful life prediction with convolutional and stacked LSTM network , 2020 .

[2]  Yanyang Zi,et al.  Switching State-Space Degradation Model With Recursive Filter/Smoother for Prognostics of Remaining Useful Life , 2019, IEEE Transactions on Industrial Informatics.

[3]  Davide Anguita,et al.  Condition-Based Maintenance of Naval Propulsion Systems with supervised Data Analysis , 2018 .

[4]  Yili Hong,et al.  Statistical Modeling of Multivariate Destructive Degradation Tests With Blocking , 2019, Technometrics.

[5]  Cong Chen,et al.  Railway turnout system RUL prediction based on feature fusion and genetic programming , 2020 .

[6]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  Brigitte Chebel-Morello,et al.  Remaining useful life estimation based on discriminating shapelet extraction , 2015, Reliab. Eng. Syst. Saf..

[9]  Geoffrey E. Hinton,et al.  Lookahead Optimizer: k steps forward, 1 step back , 2019, NeurIPS.

[10]  Rongrong Ying,et al.  Remaining useful life estimation with multiple local similarities , 2020, Eng. Appl. Artif. Intell..

[11]  Martin Bouchard,et al.  Multichannel recursive-least-square algorithms and fast-transversal-filter algorithms for active noise control and sound reproduction systems , 2000, IEEE Trans. Speech Audio Process..

[12]  Hongwen He,et al.  A Double-Scale, Particle-Filtering, Energy State Prediction Algorithm for Lithium-Ion Batteries , 2018, IEEE Transactions on Industrial Electronics.

[13]  Aaron Klein,et al.  Auto-sklearn: Efficient and Robust Automated Machine Learning , 2019, Automated Machine Learning.

[14]  Peter W. Tse,et al.  A multi-sensor approach to remaining useful life estimation for a slurry pump , 2019 .

[15]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[16]  Brigitte Chebel-Morello,et al.  Direct Remaining Useful Life Estimation Based on Support Vector Regression , 2017, IEEE Transactions on Industrial Electronics.

[17]  Enrico Zio,et al.  Remaining useful life estimation in heterogeneous fleets working under variable operating conditions , 2016, Reliab. Eng. Syst. Saf..

[18]  Enrico Zio,et al.  Some Challenges and Opportunities in Reliability Engineering , 2016, IEEE Transactions on Reliability.

[19]  Ruqiang Yan,et al.  Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks , 2017, Sensors.

[20]  Enrico Zio,et al.  A Sequential Bayesian Approach for Remaining Useful Life Prediction of Dependent Competing Failure Processes , 2019, IEEE Transactions on Reliability.

[21]  E. Zio,et al.  Remaining useful life prediction for Lithium-ion batteries using fractional Brownian motion and Fruit-fly Optimization Algorithm , 2020, Measurement.

[22]  Badong Chen,et al.  A Novel Prognostic Approach for RUL Estimation With Evolving Joint Prediction of Continuous and Discrete States , 2019, IEEE Transactions on Industrial Informatics.

[23]  Liang Guo,et al.  A recurrent neural network based health indicator for remaining useful life prediction of bearings , 2017, Neurocomputing.

[24]  Xi Zhang,et al.  A data-level fusion approach for degradation modeling and prognostic analysis under multiple failure modes , 2018 .

[25]  Nagi Gebraeel,et al.  Multistream sensor fusion-based prognostics model for systems with single failure modes , 2017, Reliab. Eng. Syst. Saf..

[26]  Xi Zhang,et al.  Integration of Data-Level Fusion Model and Kernel Methods for Degradation Modeling and Prognostic Analysis , 2018, IEEE Transactions on Reliability.

[27]  Peng-Bo Zhang,et al.  A Novel AdaBoost Framework With Robust Threshold and Structural Optimization , 2018, IEEE Transactions on Cybernetics.

[28]  Joeri Van Mierlo,et al.  Random forest regression for online capacity estimation of lithium-ion batteries , 2018, Applied Energy.

[29]  Maxim A. Dulebenets,et al.  Application of Evolutionary Computation for Berth Scheduling at Marine Container Terminals: Parameter Tuning Versus Parameter Control , 2018, IEEE Transactions on Intelligent Transportation Systems.

[30]  Axel Barrau,et al.  The Invariant Extended Kalman Filter as a Stable Observer , 2014, IEEE Transactions on Automatic Control.

[31]  Jun Wu,et al.  Machine Health Monitoring Using Adaptive Kernel Spectral Clustering and Deep Long Short-Term Memory Recurrent Neural Networks , 2019, IEEE Transactions on Industrial Informatics.

[32]  Sandeep Kumar,et al.  A novel soft computing method for engine RUL prediction , 2017, Multimedia Tools and Applications.

[33]  Antonio Bernardo Sánchez,et al.  A New Predictive Model Based on the ABC Optimized Multivariate Adaptive Regression Splines Approach for Predicting the Remaining Useful Life in Aircraft Engines , 2016 .

[34]  Zhi-Sheng Ye,et al.  RUL Prediction of Deteriorating Products Using an Adaptive Wiener Process Model , 2017, IEEE Transactions on Industrial Informatics.

[35]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[36]  Stefano Soatto,et al.  Information Dropout: Learning Optimal Representations Through Noisy Computation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[38]  C. Tenreiro,et al.  On the automatic selection of the tuning parameter appearing in certain families of goodness-of-fit tests , 2019, Journal of Statistical Computation and Simulation.

[39]  Wei Zhao,et al.  Remaining useful life prediction using multi-scale deep convolutional neural network , 2020, Appl. Soft Comput..

[40]  Davide Anguita,et al.  Condition-based maintenance of naval propulsion systems: Data analysis with minimal feedback , 2018, Reliab. Eng. Syst. Saf..

[41]  Jianbo Yu,et al.  State of health prediction of lithium-ion batteries: Multiscale logic regression and Gaussian process regression ensemble , 2018, Reliab. Eng. Syst. Saf..

[42]  Yanyang Zi,et al.  A Two-Stage Data-Driven-Based Prognostic Approach for Bearing Degradation Problem , 2016, IEEE Transactions on Industrial Informatics.

[43]  Yong Zhang,et al.  A recurrent neural network approach for remaining useful life prediction utilizing a novel trend features construction method , 2019, Measurement.

[44]  Eric Sanlaville,et al.  State estimation of discrete event systems for RUL prediction issue , 2017, Int. J. Prod. Res..

[45]  Wennian Yu,et al.  Remaining useful life estimation using a bidirectional recurrent neural network based autoencoder scheme , 2019, Mechanical Systems and Signal Processing.

[46]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[47]  David He,et al.  Using Deep Learning-Based Approach to Predict Remaining Useful Life of Rotating Components , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[48]  Bin Liang,et al.  Remaining useful life prediction of aircraft engine based on degradation pattern learning , 2017, Reliab. Eng. Syst. Saf..

[49]  Aaron Klein,et al.  Efficient and Robust Automated Machine Learning , 2015, NIPS.

[50]  Tao Yuan,et al.  A Bayesian approach to modeling two-phase degradation using change-point regression , 2015, Reliab. Eng. Syst. Saf..

[51]  Víctor Gómez,et al.  Wiener–Kolmogorov Filtering and Smoothing for Multivariate Series With State–Space Structure , 2007 .

[52]  Noureddine Zerhouni,et al.  A New Multivariate Approach for Prognostics Based on Extreme Learning Machine and Fuzzy Clustering , 2015, IEEE Transactions on Cybernetics.

[53]  Weiwen Peng,et al.  Improved trajectory similarity-based approach for turbofan engine prognostics , 2019, Journal of Mechanical Science and Technology.

[54]  Carl E. Rasmussen,et al.  Gaussian Processes for Machine Learning (GPML) Toolbox , 2010, J. Mach. Learn. Res..

[55]  Yu Zheng,et al.  An ensemble framework based on convolutional bi-directional LSTM with multiple time windows for remaining useful life estimation , 2020, Comput. Ind..