Practical options for selecting data-driven or physics-based prognostics algorithms with reviews

This paper is to provide practical options for prognostics so that beginners can select appropriate methods for their fields of application. To achieve this goal, several popular algorithms are first reviewed in the data-driven and physics-based prognostics methods. Each algorithm’s attributes and pros and cons are analyzed in terms of model definition, model parameter estimation and ability to handle noise and bias in data. Fatigue crack growth examples are then used to illustrate the characteristics of different algorithms. In order to suggest a suitable algorithm, several studies are made based on the number of data sets, the level of noise and bias, availability of loading and physical models, and complexity of the damage growth behavior. Based on the study, it is concluded that the Gaussian process is easy and fast to implement, but works well only when the covariance function is properly defined. The neural network has the advantage in the case of large noise and complex models but only with many training data sets. The particle filter and Bayesian method are superior to the former methods because they are less affected by noise and model complexity, but work only when physical model and loading conditions are available.

[1]  Michael Pecht,et al.  A comparative review of prognostics-based reliability methods for Lithium batteries , 2011, 2011 Prognostics and System Health Managment Confernece.

[2]  Luren Yang,et al.  An Evaluation of Confidence Bound Estimation Methods for Neural Networks , 2002, Advances in Computational Intelligence and Learning.

[3]  Michigan.,et al.  Estimating photometric redshifts with artificial neural networks , 2002, astro-ph/0203250.

[4]  Ashok Srivastava,et al.  Stable and Efficient Gaussian Process Calculations , 2009, J. Mach. Learn. Res..

[5]  Jianhua Z. Huang,et al.  A full scale approximation of covariance functions for large spatial data sets , 2012 .

[6]  Wee Ser,et al.  Probabilistic neural-network structure determination for pattern classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[7]  G. Vachtsevanos,et al.  Reasoning about uncertainty in prognosis: a confidence prediction neural network approach , 2005, NAFIPS 2005 - 2005 Annual Meeting of the North American Fuzzy Information Processing Society.

[8]  C. Lee Giles,et al.  What Size Neural Network Gives Optimal Generalization? Convergence Properties of Backpropagation , 1998 .

[9]  Kai Goebel,et al.  Comparison of prognostic algorithms for estimating remaining useful life of batteries , 2009 .

[10]  M.M. Gupta,et al.  Memetic Differential Evolution Trained Neural Networks For Nonlinear System Identification , 2008, 2008 IEEE Region 10 and the Third international Conference on Industrial and Information Systems.

[11]  A. P. Dawid,et al.  Regression and Classification Using Gaussian Process Priors , 2009 .

[12]  Andrew Gordon Wilson,et al.  Gaussian Process Covariance Kernels for Pattern Discovery and Extrapolation , 2013, ArXiv.

[13]  Nazri Mohd Nawi,et al.  An Improved Conjugate Gradient Based Learning Algorithm for Back Propagation Neural Networks , 2008 .

[14]  Weicheng Cui,et al.  An engineering model of fatigue crack growth under variable amplitude loading , 2008 .

[15]  Mark J. Schervish,et al.  Nonstationary Covariance Functions for Gaussian Process Regression , 2003, NIPS.

[16]  J. Leo van Hemmen,et al.  Accelerating backpropagation through dynamic self-adaptation , 1996, Neural Networks.

[17]  Haiyan Lu,et al.  Multi-step forecasting for wind speed using a modified EMD-based artificial neural network model , 2012 .

[18]  Christopher K. I. Williams.  Computing with Infinite Networks , 1996, NIPS.

[19]  Lei Xu,et al.  Health management based on fusion prognostics for avionics systems , 2011 .

[20]  Kevin Kam Fung Yuen,et al.  Toward a Hybrid Approach of Primitive Cognitive Network Process and Particle Swarm Optimization Neural Network for Forecasting , 2013, ITQM.

[21]  Jay Lee,et al.  Prognostics and health management design for rotary machinery systems—Reviews, methodology and applications , 2014 .

[22]  Wu Yunxin,et al.  Fault Diagnosis of Rolling Element Bearing Based on Vibration Frequency Analysis , 2011, 2011 Third International Conference on Measuring Technology and Mechatronics Automation.

[23]  Robert A. Jacobs,et al.  Methods For Combining Experts' Probability Assessments , 1995, Neural Computation.

[24]  N. Balakrishnan,et al.  Residual life estimation based on bivariate non-stationary gamma degradation process , 2015 .

[25]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[26]  G. Kitagawa.  Non-Gaussian State—Space Modeling of Nonstationary Time Series , 1987 .

[27]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[28]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[29]  George Chryssolouris,et al.  Confidence interval prediction for neural network models , 1996, IEEE Trans. Neural Networks.

[30]  Yichuang Sun,et al.  Wavelet neural network approach for fault diagnosis of analogue circuits , 2004 .

[31]  Shawki A. Abouel-seoud,et al.  Robust Prognostics Concept for Gearbox with Artificially Induced Gear Crack Utilizing Acoustic Emission , 2011 .

[32]  Daniel Svozil,et al.  Introduction to multi-layer feed-forward neural networks , 1997 .

[33]  Bhaskar Saha,et al.  An Adaptive Recurrent Neural Network for Remaining Useful Life Prediction of Lithium-ion Batteries , 2010 .

[34]  Wei Liang,et al.  Remaining useful life prediction of lithium-ion battery with unscented particle filter technique , 2013, Microelectron. Reliab..

[35]  Léon Personnaz,et al.  Construction of confidence intervals for neural networks based on least squares estimation , 2000, Neural Networks.

[36]  Bin Zhang,et al.  Application of Blind Deconvolution Denoising in Failure Prognosis , 2009, IEEE Transactions on Instrumentation and Measurement.

[37]  Marek Krawczuk,et al.  Improvement of damage detection methods based on experimental modal parameters , 2011 .

[38]  Enrico Zio,et al.  A data-driven fuzzy approach for predicting the remaining useful life in dynamic failure scenarios of a nuclear system , 2010, Reliab. Eng. Syst. Saf..

[39]  D. Rubin.  Using the SIR algorithm to simulate posterior distributions , 1988 .

[40]  Matthias W. Seeger,et al.  Gaussian Processes For Machine Learning , 2004, Int. J. Neural Syst..

[41]  Joo-Ho Choi,et al.  A Comparison Study of Methods for Parameter Estimation in the Physics-based Prognostics , 2012, Annual Conference of the PHM Society.

[42]  Yu Peng,et al.  Data-driven prognostics for lithium-ion battery based on Gaussian Process Regression , 2012, Proceedings of the IEEE 2012 Prognostics and System Health Management Conference (PHM-2012 Beijing).

[43]  Yongqian Liu,et al.  Wind power grouping forecasts and its uncertainty analysis using optimized relevance vector machine , 2013 .

[44]  Carlos Henggeler Antunes,et al.  Comparison of a genetic algorithm and simulated annealing for automatic neural network ensemble development , 2013, Neurocomputing.

[45]  A. Chattopadhyay,et al.  Gaussian Process Time Series Model for Life Prognosis of Metallic Structures , 2009 .

[46]  Robert B. Gramacy,et al.  Cases for the nugget in modeling computer experiments , 2010, Statistics and Computing.

[47]  O. A. Hodhod,et al.  Developing an artificial neural network model to evaluate chloride diffusivity in high performance concrete , 2013 .

[48]  Yan-Ping Wang,et al.  Bayesian inference and prediction analysis of the power law process based on a natural conjugate prior , 2015 .

[49]  Raphael T. Haftka,et al.  Reducing Uncertainty in Damage Growth Properties by Structural Health Monitoring , 2009 .

[50]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[51]  Michael Osterman,et al.  Prognostics of Lithium-ion Batteries using Extended Kalman Filtering , 2011 .

[52]  Saeid Nahavandi,et al.  Quantifying uncertainties of neural network-based electricity price forecasts , 2013 .

[53]  Donghua Zhou,et al.  A Wiener-process-based degradation model with a recursive filter algorithm for remaining useful life estimation , 2013 .

[54]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[55]  Sankaran Mahadevan,et al.  Quantitative model validation techniques: New insights , 2012, Reliab. Eng. Syst. Saf..

[56]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[57]  Bart L. M. Happel,et al.  Design and evolution of modular neural network architectures , 1994, Neural Networks.

[58]  Victor Giurgiutiu,et al.  Current issues in vibration-based fault diagnostics and prognostics , 2002, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[59]  Hongxing Li,et al.  Fuzzy Neural Network Theory and Application , 2004, Series in Machine Perception and Artificial Intelligence.

[60]  R. D. Veaux,et al.  Prediction intervals for neural networks via nonlinear regression , 1998 .

[61]  Nam H. Kim,et al.  Identification of correlated damage parameters under noise and bias using Bayesian inference , 2011 .

[62]  Lei Deng,et al.  Fault diagnosis for a wind turbine transmission system based on manifold learning and Shannon wavelet support vector machine , 2014 .

[63]  É MikaelBoden.  A guide to recurrent neural networks and backpropagation , 2001 .

[64]  Neil D. Lawrence,et al.  Fast Sparse Gaussian Process Methods: The Informative Vector Machine , 2002, NIPS.

[65]  Meng Chao,et al.  Neural network ensembles based on copula methods and Distributed Multiobjective Central Force Optimization algorithm , 2014, Eng. Appl. Artif. Intell..

[66]  Daniel Hissel,et al.  Proton exchange membrane fuel cell degradation prediction based on Adaptive Neuro-Fuzzy Inference Systems . , 2014 .

[67]  Peter Challenor,et al.  Computational Statistics and Data Analysis the Effect of the Nugget on Gaussian Process Emulators of Computer Models , 2022 .

[68]  P. C. Paris,et al.  A Critical Analysis of Crack Propagation Laws , 1963 .

[69]  T. Bayes.  An essay towards solving a problem in the doctrine of chances , 2003 .

[70]  David Mackay,et al.  Gaussian Processes - A Replacement for Supervised Neural Networks? , 1997 .

[71]  Jerry Nedelman,et al.  Book review: “Bayesian Data Analysis,” Second Edition by A. Gelman, J.B. Carlin, H.S. Stern, and D.B. Rubin Chapman & Hall/CRC, 2004 , 2005, Comput. Stat..

[72]  A. Gelfand,et al.  On Markov Chain Monte Carlo Acceleration , 1994 .

[73]  Mikael Bodén,et al.  A guide to recurrent neural networks and backpropagation , 2001 .

[74]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[75]  Xuefei Guan,et al.  Entropy-based probabilistic fatigue damage prognosis and algorithmic performance comparison , 2009 .

[76]  Yoshihiro Deguchi,et al.  Applications of laser diagnostics to thermal power plants and engines , 2014 .

[77]  George J. Vachtsevanos,et al.  A Particle Filtering Approach for On-Line Failure Prognosis in a Planetary Carrier Plate , 2007, Int. J. Fuzzy Log. Intell. Syst..

[78]  G.A. Rovithakis,et al.  A hybrid neural network/genetic algorithm approach to optimizing feature extraction for signal classification , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[79]  T. Higuchi.  Monte carlo filter using the genetic algorithm operators , 1997 .

[80]  Ting-wen Xing,et al.  Particle filter for state and parameter estimation in passive ranging , 2009, 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems.

[81]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[82]  Donghua Zhou,et al.  Remaining useful life estimation - A review on the statistical data driven approaches , 2011, Eur. J. Oper. Res..

[83]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[84]  Ying Peng,et al.  A novel method for online health prognosis of equipment based on hidden semi-Markov model using sequential Monte Carlo methods , 2012 .

[85]  J. Tinsley Oden,et al.  Virtual model validation of complex multiscale systems: Applications to nonlinear elastostatics , 2013 .

[86]  V. Sugumaran,et al.  Fault diagnostics of roller bearing using kernel based neighborhood score multi-class support vector machine , 2008, Expert Syst. Appl..

[87]  Harris Drucker,et al.  Boosting and Other Ensemble Methods , 1994, Neural Computation.

[88]  Linxia Liao,et al.  Review of Hybrid Prognostics Approaches for Remaining Useful Life Prediction of Engineered Systems, and an Application to Battery Life Prediction , 2014, IEEE Transactions on Reliability.

[89]  Xiao Liu,et al.  Condition-based maintenance for continuously monitored degrading systems with multiple failure modes , 2013 .

[90]  Daming Lin,et al.  A review on machinery diagnostics and prognostics implementing condition-based maintenance , 2006 .

[91]  Jeong-Soo Park,et al.  Sequential Monte Carlo filters for abruptly changing state estimation , 2011 .

[92]  Fabio Tozeto Ramos,et al.  A Sparse Covariance Function for Exact Gaussian Process Inference in Large Datasets , 2009, IJCAI.

[93]  Leonardo Franco,et al.  Neural Network Architecture Selection: Can Function Complexity Help? , 2009, Neural Processing Letters.

[94]  Joo-Ho Choi,et al.  Efficient reliability analysis based on Bayesian framework under input variable and metamodel uncertainties , 2012 .

[95]  Joo-Ho Choi,et al.  MCMC Approach for Parameter Estimation in the Structural Analysis and Prognosis , 2010 .

[96]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[97]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[98]  Antoine Grall,et al.  A condition-based maintenance policy for stochastically deteriorating systems , 2002, Reliab. Eng. Syst. Saf..

[99]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[100]  Jay Lee,et al.  Robust performance degradation assessment methods for enhanced rolling element bearing prognostics , 2003, Adv. Eng. Informatics.

[101]  Shih-Wei Lin,et al.  Optimization of Back-Propagation Network Using Simulated Annealing Approach , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[102]  Nathan Intrator,et al.  Optimal ensemble averaging of neural networks , 1997 .

[103]  D. An,et al.  In-Situ Monitoring and Prediction of Progressive Joint Wear Using Bayesian Statistics , 2010 .

[104]  Wilson Wang,et al.  Enhanced fuzzy-filtered neural networks for material fatigue prognosis , 2013, Appl. Soft Comput..

[105]  Jan Lundberg,et al.  Remaining useful life prediction of grinding mill liners using an artificial neural network , 2013 .

[106]  Nam H. Kim,et al.  Bayesian Approach for Par ameter Estimation in the Str uctur al Analysis and Prognosis , 2010 .

[107]  Raphael T. Haftka,et al.  Using a Simple Crack Growth Model in Predicting Remaining Useful Life , 2012 .

[108]  Carlo Di Bello,et al.  Analysis of an associative memory neural network for pattern identification in gene expression data , 2001, BIOKDD.

[109]  Jay Lee,et al.  A review on prognostics and health monitoring of Li-ion battery , 2011 .

[110]  K. Gnana Sheela,et al.  Review on Methods to Fix Number of Hidden Neurons in Neural Networks , 2013 .

[111]  K. F. Martin,et al.  A review by discussion of condition monitoring and fault diagnosis in machine tools , 1994 .

[112]  Jian Ma,et al.  A new neural network model for the state-of-charge estimation in the battery degradation process , 2014 .

[113]  Robert X. Gao,et al.  Mechanical Systems and Signal Processing Approximate Entropy as a Diagnostic Tool for Machine Health Monitoring , 2006 .

[114]  G. Storvik.  Particle filters in state space models with the presence of unknown static parameters YYYY No org found YYY , 2000 .

[115]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[116]  Amir F. Atiya,et al.  Comprehensive Review of Neural Network-Based Prediction Intervals and New Advances , 2011, IEEE Transactions on Neural Networks.

[117]  Okyay Kaynak,et al.  An algorithm for fast convergence in training neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[118]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[119]  Sankaran Mahadevan,et al.  Validation and error estimation of computational models , 2006, Reliab. Eng. Syst. Saf..

[120]  Noureddine Zerhouni,et al.  Health assessment and life prediction of cutting tools based on support vector regression , 2015, J. Intell. Manuf..

[121]  Dawn An,et al.  Prognostics 101: A tutorial for particle filter-based prognostics algorithm using Matlab , 2013, Reliab. Eng. Syst. Saf..

[122]  Norbert Jankowski,et al.  Survey of Neural Transfer Functions , 1999 .

[123]  Kishan G. Mehrotra,et al.  Forecasting the behavior of multivariate time series using neural networks , 1992, Neural Networks.

[124]  Qiang Miao,et al.  Prognostics and health monitoring for lithium-ion battery , 2011, Proceedings of 2011 IEEE International Conference on Intelligence and Security Informatics.

[125]  Darryll J. Pines,et al.  A review of vibration-based techniques for helicopter transmission diagnostics , 2005 .

[126]  A. Saltelli,et al.  Reliability Engineering and System Safety , 2008 .

[127]  Aitor Arnaiz,et al.  Ball bearing damage detection using traditional signal processing algorithms , 2013, IEEE Instrumentation & Measurement Magazine.

[128]  Raphael T. Haftka,et al.  Technical Notes Uncertainty Identification of Damage Growth Parameters Using Nonlinear Regression , 2011 .

[129]  Gomes de Freitas,et al.  Bayesian methods for neural networks , 2000 .

[130]  M.H. Azarian,et al.  Failure prognostics of multilayer ceramic capacitors in temperature-humidity-bias conditions , 2008, 2008 International Conference on Prognostics and Health Management.

[131]  Jui-Fang Chang,et al.  PARTICLE SWARM OPTIMIZATION BASED ON BACK PROPAGATION NETWORK FORECASTING EXCHANGE RATES , 2011 .

[132]  Nando de Freitas,et al.  The Unscented Particle Filter , 2000, NIPS.

[133]  K. Goebel,et al.  Multiple damage progression paths in model-based prognostics , 2011, 2011 Aerospace Conference.

[134]  Joo-Ho Choi,et al.  Improved MCMC Method for Parameter Estimation Based on Marginal Probability Density Function , 2011 .

[135]  Amine Bermak,et al.  Gaussian process for nonstationary time series prediction , 2004, Comput. Stat. Data Anal..

[136]  Dumitru Ostafe.  Neural Network Hidden Layer Number Determination Using Pattern Recognition Techniques , 2005 .