More effective prognostics with elbow point detection and deep learning

Abstract Prior to failure, most systems exhibit signs of changed characteristics. The early detection of this change is important to remaining useful life estimation. To have the ability to detect the inflection point or “elbow point” of an asset, i.e. the point of the degradation curve that marks the transition from nominal to faulty condition, can enable more sophisticated prognostics because this divide and conquer tactic allows the prediction to focus on the window before failure when significant changes are being expected. In this work, we compare prognostics with and without change point detection. We use different recurrent neural network techniques (standard recurrent neural network, long short-term memory and gated recurrent unit) to find the elbow point location. The actual estimation of the remaining time to failure is based on the echo state network, a state-of-the-art approach in prognostics. Two different experiments are performed on simulated data obtained from NASA Ames prognostics repository. We first compare the performance of the elbow point detectors based on recurrent neural networks against three baseline models: the Z-test, multi-layer perceptron and random forests. Results indicate that recurrent neural networks can outperform the baseline approaches. In the second experiment, the best elbow detection model, the gated recurrent unit, is integrated within an echo state network, with a significant increase in overall performance in terms of remaining useful life estimation.

[1]  Jürgen Schmidhuber,et al.  Learning Complex, Extended Sequences Using the Principle of History Compression , 1992, Neural Computation.

[2]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[3]  Nigel Collier,et al.  Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation , 2012, Neural Networks.

[4]  Zaïd Harchaoui,et al.  A regularized kernel-based approach to unsupervised audio segmentation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  G. Lorden PROCEDURES FOR REACTING TO A CHANGE IN DISTRIBUTION , 1971 .

[6]  Alexander G. Tartakovsky,et al.  Asymptotic Optimality of Change-Point Detection Schemes in General Continuous-Time Models , 2006 .

[7]  Enrico Zio,et al.  Predicting time series of railway speed restrictions with time-dependent machine learning techniques , 2013, Expert Syst. Appl..

[8]  Dawn An,et al.  Practical options for selecting data-driven or physics-based prognostics algorithms with reviews , 2015, Reliab. Eng. Syst. Saf..

[9]  Abhinav Saxena,et al.  Performance Benchmarking and Analysis of Prognostic Methods for CMAPSS Datasets , 2020, International Journal of Prognostics and Health Management.

[10]  Linxia Liao,et al.  Review of Hybrid Prognostics Approaches for Remaining Useful Life Prediction of Engineered Systems, and an Application to Battery Life Prediction , 2014, IEEE Transactions on Reliability.

[11]  Hao Chen,et al.  Sequential change-point detection based on nearest neighbors , 2016, The Annals of Statistics.

[12]  Hervé Bourlard,et al.  Generalization and Parameter Estimation in Feedforward Netws: Some Experiments , 1989, NIPS.

[13]  E. S. Page CONTINUOUS INSPECTION SCHEMES , 1954 .

[14]  Adel M. Alimi,et al.  PSO-based analysis of Echo State Network parameters for time series forecasting , 2017, Appl. Soft Comput..

[15]  S. Czesla,et al.  A posteriori noise estimation in variable data sets: With applications to spectra and light curves , 2017, 1712.02226.

[16]  V. Veeravalli,et al.  General Asymptotic Bayesian Theory of Quickest Change Detection , 2005 .

[17]  Rui Xiong,et al.  A LSTM-RNN method for the lithuim-ion battery remaining useful life prediction , 2017, 2017 Prognostics and System Health Management Conference (PHM-Harbin).

[18]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[19]  Dong Dong,et al.  Life prediction of jet engines based on LSTM-recurrent neural networks , 2017, 2017 Prognostics and System Health Management Conference (PHM-Harbin).

[20]  Xiao Li,et al.  A Novel SOH Prediction Framework for the Lithium-ion Battery Using Echo State Network , 2014, ICONIP.

[21]  Tom Fawcett,et al.  ROC graphs with instance-varying costs , 2006, Pattern Recognit. Lett..

[22]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[23]  A. Shiryaev On Optimum Methods in Quickest Detection Problems , 1963 .

[24]  Ahmed Zakariae Hinchi,et al.  A deep long-short-term-memory neural network for lithium-ion battery prognostics , 2018 .

[25]  Peter W. Tse,et al.  Prediction of Machine Deterioration Using Vibration Based Fault Trends and Recurrent Neural Networks , 1999 .

[26]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[27]  Xing Xie,et al.  Learning transportation mode from raw gps data for geographic applications on the web , 2008, WWW.

[28]  Fred L. Collopy,et al.  Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons , 1992 .

[29]  Jeffrey L. Elman,et al.  Distributed Representations, Simple Recurrent Networks, and Grammatical Structure , 1991, Mach. Learn..

[30]  Michèle Basseville,et al.  Detecting changes in signals and systems - A survey , 1988, Autom..

[31]  F.O. Heimes,et al.  Recurrent neural networks for remaining useful life estimation , 2008, 2008 International Conference on Prognostics and Health Management.

[32]  A. Aue,et al.  Break detection in the covariance structure of multivariate time series models , 2009, 0911.3796.

[33]  Piotr Fryzlewicz,et al.  Multiple‐change‐point detection for high dimensional time series via sparsified binary segmentation , 2015, 1611.08639.

[34]  Hagbae Kim,et al.  CNN and GRU combination scheme for Bearing Anomaly Detection in Rotating Machinery Health Monitoring , 2018, 2018 1st IEEE International Conference on Knowledge Innovation and Invention (ICKII).

[35]  E. Zio,et al.  Nuclear reactor dynamics on-line estimation by Locally Recurrent Neural Networks , 2009 .

[36]  N. Vayatis,et al.  Selective review of offline change point detection methods , 2019 .

[37]  Young-Koo Lee,et al.  Comprehensive Context Recognizer Based on Multimodal Sensors in a Smartphone , 2012, Sensors.

[38]  Enrico Zio,et al.  Ensemble of optimized echo state networks for remaining useful life prediction , 2017, Neurocomputing.

[39]  V. Moskvina,et al.  An Algorithm Based on Singular Spectrum Analysis for Change-Point Detection , 2003 .

[40]  Gregory Levitin,et al.  Robust recurrent neural network modeling for software fault detection and correction prediction , 2007, Reliab. Eng. Syst. Saf..

[41]  Mohamad T. Musavi,et al.  On the Generalization Ability of Neural Network Classifiers , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Yoshinobu Kawahara,et al.  Change-Point Detection in Time-Series Data Based on Subspace Identification , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[43]  Dirk P. Kroese,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning , 2004 .

[44]  G. Moustakides Optimal stopping times for detecting changes in distributions , 1986 .

[45]  Xing Xie,et al.  Understanding transportation modes based on GPS data for web applications , 2010, TWEB.

[46]  Jan Svec,et al.  On Using Stateful LSTM Networks for Key-Phrase Detection , 2019, TSD.

[47]  Chris D. Nugent,et al.  Evaluation of Prompted Annotation of Activity Data Recorded from a Smart Phone , 2014, Sensors.

[48]  Hagbae Kim,et al.  Stacked Convolutional Bidirectional LSTM Recurrent Neural Network for Bearing Anomaly Detection in Rotating Machinery Diagnostics , 2018, 2018 1st IEEE International Conference on Knowledge Innovation and Invention (ICKII).

[49]  Daming Lin,et al.  A review on machinery diagnostics and prognostics implementing condition-based maintenance , 2006 .

[50]  Fei Gao,et al.  Data-driven proton exchange membrane fuel cell degradation predication through deep learning method , 2018, Applied Energy.

[51]  Hao Wang,et al.  Detecting Transportation Modes Using Deep Neural Network , 2017, IEICE Trans. Inf. Syst..

[52]  Enrico Zio,et al.  Predicting component reliability and level of degradation with complex-valued neural networks , 2014, Reliab. Eng. Syst. Saf..

[53]  M. Basseville,et al.  Sequential Analysis: Hypothesis Testing and Changepoint Detection , 2014 .

[54]  Abhinav Saxena,et al.  Damage propagation modeling for aircraft engine run-to-failure simulation , 2008, 2008 International Conference on Prognostics and Health Management.

[55]  T.K. Das,et al.  A Multiscale Bayesian SPRT Approach for Online Process Monitoring , 2008, IEEE Transactions on Semiconductor Manufacturing.

[56]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[57]  Diane J. Cook,et al.  A survey of methods for time series change point detection , 2017, Knowledge and Information Systems.

[58]  K. Goebel,et al.  Metrics for evaluating performance of prognostic techniques , 2008, 2008 International Conference on Prognostics and Health Management.

[59]  Deborah Estrin,et al.  Using mobile phones to determine transportation modes , 2010, TOSN.

[60]  Joshua I. Gold,et al.  Bayesian Online Learning of the Hazard Rate in Change-Point Problems , 2010, Neural Computation.

[61]  Peter C. Kiessler,et al.  A critical look at the bathtub curve , 2003, IEEE Trans. Reliab..

[62]  Li Wei,et al.  Semi-supervised time series classification , 2006, KDD '06.

[63]  Karl Aberer,et al.  Robust Online Time Series Prediction with Recurrent Neural Networks , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[64]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[65]  Xiangnan Li,et al.  Prognostics Comparison of Lithium-Ion Battery Based on the Shallow and Deep Neural Networks Model , 2019, Energies.

[66]  Li Lin,et al.  Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network , 2016, 2016 IEEE International Conference on Aircraft Utility Systems (AUS).