Noise-dependent ranking of prognostics algorithms based on discrepancy without true damage information

In this paper, an interesting observation on the noise-dependent performance of prognostics algorithms is presented. A method of evaluating the accuracy of prognostics algorithms without having the true degradation model is proposed. This paper compares the four most widely used model-based prognostics algorithms, i.e., Bayesian method, particle filter, Extended Kalman filter, and nonlinear least squares, to illustrate the effect of random noise in data on the performance of prediction. The mean squared error (MSE) that measures the difference between the true damage size and the predicted one is used to rank the four algorithms for each dataset. We found that the randomness in the noise leads to a very different ranking of the algorithms for different datasets, even though they are all from the same damage model. In particular, even for the algorithm that has the best performance on average, poor results can be obtained for some datasets. In absence of true damage information, we propose another metric, mean squared discrepancy (MSD), which measures the difference between the prediction and the data. A correlation study between MSE and MSD indicates that MSD can be used to estimate the ranking of the four prognostics algorithms without having the true damage information. Moreover, the best algorithm selected by MSD has a high probability of also having the smallest prediction error when used for predicting beyond the last measurement. MSD can thus be particularly useful for selecting the best algorithm for predicting into the near future for a given set of measurements.

[1]  Yu Peng,et al.  Comparison of resampling algorithms for particle filter based remaining useful life estimation , 2014, 2014 International Conference on Prognostics and Health Management.

[2]  Chee Khiang Pang,et al.  Precognitive maintenance and probabilistic assessment of tool wear using particle filters , 2013, IECON 2013 - 39th Annual Conference of the IEEE Industrial Electronics Society.

[3]  Juan Garcia-Velo,et al.  Aerodynamic Parameter Estimation for High-Performance Aircraft Using Extended Kalman Filtering , 1997 .

[4]  Mahendra P. Singh,et al.  An adaptive unscented Kalman filter for tracking sudden stiffness changes , 2014 .

[5]  Jerker Nordh.  Metropolis-hastings improved particle smoother and marginalized models , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[6]  Walter Sextro,et al.  PEM fuel cell prognostics using particle filter with model parameter adaptation , 2014, 2014 International Conference on Prognostics and Health Management.

[7]  Dawn An,et al.  Prognostics 101: A tutorial for particle filter-based prognostics algorithm using Matlab , 2013, Reliab. Eng. Syst. Saf..

[8]  Michael G. Pecht,et al.  A prognostics and health management roadmap for information and electronics-rich systems , 2010, Microelectron. Reliab..

[9]  Nam H. Kim,et al.  Identification of correlated damage parameters under noise and bias using Bayesian inference , 2011 .

[10]  Jay Lee,et al.  Prognostics and health management design for rotary machinery systems—Reviews, methodology and applications , 2014 .

[11]  G. G. Garrett,et al.  On the correlation between the C and m in the paris equation for fatigue crack propagation , 1988 .

[12]  Ralph E. White,et al.  Comparison of a particle filter and other state estimation methods for prognostics of lithium-ion batteries , 2015 .

[13]  D. V. Edmonds,et al.  The relationship between the parameters C and m of Paris' law for fatigue crack growth in a low-alloy steel , 1978 .

[14]  Enrico Zio,et al.  Model-based and data-driven prognostics under different available information , 2013 .

[15]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[16]  Belkacem Ould Bouamama,et al.  Extended Kalman Filter for prognostic of Proton Exchange Membrane Fuel Cell , 2016 .

[17]  Mohinder S. Grewal,et al.  Kalman Filtering: Theory and Practice Using MATLAB , 2001 .

[18]  Sankalita Saha,et al.  Metrics for Offline Evaluation of Prognostic Performance , 2021, International Journal of Prognostics and Health Management.

[19]  Dawn An,et al.  Prognostics and Health Management of Engineering Systems: An Introduction , 2016 .

[20]  Enrico Zio,et al.  Particle filtering prognostic estimation of the remaining useful life of nonlinear components , 2011, Reliab. Eng. Syst. Saf..

[21]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[22]  Joseph Mathew,et al.  A review on prognostic techniques for non-stationary and non-linear rotating systems , 2015 .

[23]  Dawn An,et al.  Practical options for selecting data-driven or physics-based prognostics algorithms with reviews , 2015, Reliab. Eng. Syst. Saf..

[24]  Ömer G. Bilir.  The relationship between the parameters C and n of Paris' law for fatigue crack growth in a SAE 1010 steel , 1990 .

[25]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[26]  Xiaoning Jin,et al.  Lithium-ion battery state of health monitoring and remaining useful life prediction based on support vector regression-particle filter , 2014 .

[27]  Enrico Zio,et al.  Online Performance Assessment Method for a Model-Based Prognostic Approach , 2016, IEEE Transactions on Reliability.

[28]  Enrico Zio,et al.  A particle filtering and kernel smoothing-based approach for new design component prognostics , 2015, Reliab. Eng. Syst. Saf..

[29]  Tony L. Schmitz,et al.  Prediction of remaining useful life for fatigue-damaged structures using Bayesian inference , 2012 .

[30]  Noureddine Zerhouni,et al.  Prognostics of PEM fuel cell in a particle filtering framework , 2014 .

[31]  Byeng D. Youn,et al.  A generic probabilistic framework for structural health prognostics and uncertainty management , 2012 .

[32]  Shankar Sankararaman,et al.  Significance, interpretation, and quantification of uncertainty in prognostics and remaining useful life prediction , 2015 .

[33]  Joel P. Conte,et al.  A recursive Bayesian approach for fatigue damage prognosis: An experimental validation at the reliability component level , 2014 .

[34]  Donghua Zhou,et al.  Estimating Remaining Useful Life With Three-Source Variability in Degradation Modeling , 2014, IEEE Transactions on Reliability.

[35]  Susan S. Lu,et al.  Predictive condition‐based maintenance for continuously deteriorating systems , 2007, Qual. Reliab. Eng. Int..

[36]  Nam H. Kim,et al.  A cost driven predictive maintenance policy for structural airframe maintenance , 2017 .

[37]  R. V. Jategaonkar,et al.  Aerodynamic Parameter Estimation from Flight Data Applying Extended and Unscented Kalman Filter , 2010 .

[38]  David He,et al.  Lithium-ion battery life prognostic health management system using particle filtering framework , 2011 .

[39]  Joo-Ho Choi,et al.  A Comparison Study of Methods for Parameter Estimation in the Physics-based Prognostics , 2012, Annual Conference of the PHM Society.

[40]  Xiao-Sheng Si,et al.  An Adaptive Prognostic Approach via Nonlinear Degradation Modeling: Application to Battery Data , 2015, IEEE Transactions on Industrial Electronics.