Computing confidence and prediction intervals of industrial equipment degradation by bootstrapped support vector regression

Data-driven learning methods for predicting the evolution of the degradation processes affecting equipment are becoming increasingly attractive in reliability and prognostics applications. Among these, we consider here Support Vector Regression (SVR), which has provided promising results in various applications. Nevertheless, the predictions provided by SVR are point estimates whereas in order to take better informed decisions, an uncertainty assessment should be also carried out. For this, we apply bootstrap to SVR so as to obtain confidence and prediction intervals, without having to make any assumption about probability distributions and with good performance even when only a small data set is available. The bootstrapped SVR is first verified on Monte Carlo experiments and then is applied to a real case study concerning the prediction of degradation of a component from the offshore oil industry. The results obtained indicate that the bootstrapped SVR is a promising tool for providing reliable point and interval estimates, which can inform maintenance-related decisions on degrading components.

[1]  Chien-Feng Huang,et al.  A hybrid stock selection model using genetic algorithms and support vector regression , 2012, Appl. Soft Comput..

[2]  James Davidson,et al.  Implementing the wild bootstrap using a two-point distribution☆ , 2007 .

[3]  Enrico Zio,et al.  Reliability prediction of oil wells by support vector machine with particle swarm optimization for variable selection and hyperparameter tuning , 2011 .

[4]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[5]  Enrique López Droguett,et al.  Prediction of sea surface temperature in the tropical Atlantic by support vector machines , 2013, Comput. Stat. Data Anal..

[6]  Davide Anguita,et al.  Evaluating the Generalization Ability of Support Vector Machines through the Bootstrap , 2000, Neural Processing Letters.

[7]  Ping-Feng Pai,et al.  System reliability forecasting by support vector machines with genetic algorithms , 2006, Math. Comput. Model..

[8]  Francisco Cribari-Neto,et al.  Heteroskedasticity-consistent interval estimators , 2009 .

[9]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[10]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[11]  Enrico Zio,et al.  Reliability engineering: Old problems and new challenges , 2009, Reliab. Eng. Syst. Saf..

[12]  Jian-Bo Yang,et al.  Feature selection for support vector regression using probabilistic prediction , 2010, KDD.

[13]  Tom Heskes,et al.  Practical Confidence and Prediction Intervals , 1996, NIPS.

[14]  Ming-Wei Chang,et al.  Load Forecasting Using Support Vector Machines: A Study on EUNITE Competition 2001 , 2004, IEEE Transactions on Power Systems.

[15]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[16]  Farookh Khadeer Hussain,et al.  Support vector regression with chaos-based firefly algorithm for stock market price forecasting , 2013, Appl. Soft Comput..

[17]  Yusuke Tanaka,et al.  Cross-Validation, Bootstrap, and Support Vector Machines , 2011, Adv. Artif. Neural Syst..

[18]  Regina Y. Liu.  Bootstrap Procedures under some Non-I.I.D. Models , 1988 .

[19]  Yuping Zhang,et al.  The kinetics of carbonate scaling—application for the prediction of downhole carbonate scaling , 2001 .

[20]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[21]  E. Zio,et al.  A study of the bootstrap method for estimating the accuracy of artificial neural networks in predicting nuclear transient processes , 2006, IEEE Transactions on Nuclear Science.

[22]  Francisco Cribari-Neto,et al.  Asymptotic inference under heteroskedasticity of unknown form , 2004, Comput. Stat. Data Anal..

[23]  Anne Neville,et al.  Calcium carbonate scale formation-assessing the initial stages of precipitation and deposition , 2005 .

[24]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[25]  Enrico Zio,et al.  Failure Prediction of Oil Wells by Support Vector Regression with Variable Selection, Hyperparameter Tuning and Uncertainty Analysis , 2013 .

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  Enrico Zio,et al.  Quantifying uncertainties in the estimation of safety parameters by using bootstrapped artificial neural networks , 2008 .

[28]  Gordon Michael Graham,et al.  The effect of temperature and pressure on oilfield scale formation , 2002 .

[29]  Hsuan-Tien Lin.  A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[30]  Enrico Zio,et al.  Quantitative functional failure analysis of a thermal-hydraulic passive system by means of bootstrapped Artificial Neural Networks , 2010 .

[31]  Elizabeth A. Peck,et al.  Introduction to Linear Regression Analysis , 2001 .

[32]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[33]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[34]  Enrico Zio,et al.  A particle swarm‐optimized support vector machine for reliability prediction , 2012, Qual. Reliab. Eng. Int..

[35]  J. Mahjoobi,et al.  Prediction of significant wave height using regressive support vector machines , 2009 .

[36]  JinXing Che,et al.  Optimal training subset in a support vector regression electric load forecasting model , 2012, Appl. Soft Comput..

[37]  Enrico Zio,et al.  A model based on bootstrapped neural networks for computing the maximum fuel cladding temperature in an Rmbk-1500 nuclear reactor accident , 2008 .

[38]  Enrico Zio,et al.  NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment , 2013, Expert Syst. Appl..

[39]  Johan A. K. Suykens,et al.  Approximate Confidence and Prediction Intervals for Least Squares Support Vector Regression , 2011, IEEE Transactions on Neural Networks.

[40]  Enrico Zio,et al.  Failure and reliability prediction by support vector machines regression of time series data , 2011, Reliab. Eng. Syst. Saf..

[41]  Asit P. Basu,et al.  Statistical Methods for the Reliability of Repairable Systems , 2000 .