Industrial time series forecasting based on improved Gaussian process regression

Industrial processes often include shifting operating phases and dynamics, and system uncertainty. Industrial time series data may obey different distributions because of the time-varying characteristic. Therefore, a single global model cannot describe the local characteristics of multiple distributions. In this work, a hybrid GMM-IGPR model is proposed to solve this kind of time series prediction problem by using an improved Gaussian process regression (GPR) based on Gaussian mixture model (GMM) and a variant of the basic particles swarm optimization (PSO). In a first treatment to the time series, different distributions of the original dataset are characterized by adopting the GMM as a cluster method. Then, multiple localized GPR models are built to characterize the different properties between inputs and output within various clusters. In order to optimize the proposed algorithms, this paper utilizes the DEPSO which introduces differential evolution (DE) operator into the basic PSO algorithm to estimate hyperparameters of the GPR model, instead of using the traditional conjugate gradient (CG) method. Lastly, the Bayesian inference strategy is used to estimate the posterior probabilities of the test data with respect to different clusters. The various localized GPR models are integrated through these posterior probabilities as the weightings so that a global predictive model is developed for the final prediction. The effectiveness of the proposed algorithm is verified by means of a numerical example and a real industrial winding process. Statistical tests of experimental results compared with other popular prediction models demonstrate the good performance of the proposed model.

[1]  Giorgio Graditi,et al.  Comparative analysis of data-driven methods online and offline trained to the forecasting of grid-connected photovoltaic plant production , 2017 .

[2]  Wang Baojian,et al.  Multi-mode acid concentration prediction models of cold-rolled strip steel pickling process , 2014 .

[3]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[4]  Yi Liu,et al.  Integrated soft sensor using just-in-time support vector regression and probabilistic analysis for quality prediction of multi-grade processes , 2013 .

[5]  Shin Ishii,et al.  On-line EM Algorithm for the Normalized Gaussian Network , 2000, Neural Computation.

[6]  P. S. Heyns,et al.  An integrated Gaussian process regression for prediction of remaining useful life of slow speed bearings based on acoustic emission , 2017 .

[7]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[8]  James Kennedy,et al.  The particle swarm: social adaptation of knowledge , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[9]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[10]  Witold Pedrycz,et al.  Hybrid Neural Prediction and Optimized Adjustment for Coke Oven Gas System in Steel Industry , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Douglas A. Reynolds,et al.  Gaussian Mixture Models , 2018, Encyclopedia of Biometrics.

[12]  Stan Lipovetsky,et al.  Clusterability assessment for Gaussian mixture models , 2015, Appl. Math. Comput..

[13]  Biao Huang,et al.  Robust Gaussian process modeling using EM algorithm , 2016 .

[14]  Wei Sun,et al.  Factor analysis and forecasting of CO2 emissions in Hebei, using extreme learning machine based on particle swarm optimization , 2017 .

[15]  Guo Jia-qi.  Intelligent analysis model of landslide displacement time series based on coupling PSO-GPR , 2011 .

[16]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[17]  Ming Liu,et al.  A Rolling Grey Model Optimized by Particle Swarm Optimization in Economic Prediction , 2016, Comput. Intell..

[18]  Yanbin Yuan,et al.  Wind power prediction using hybrid autoregressive fractionally integrated moving average and least square support vector machine , 2017 .

[19]  T. Bastogne,et al.  Application of subspace methods to the identification of a winding process , 1997, 1997 European Control Conference (ECC).

[20]  Jie Yu,et al.  Online quality prediction of nonlinear and non-Gaussian chemical processes with shifting dynamics using finite mixture model based Gaussian process regression approach , 2012 .

[21]  Vadlamani Ravi,et al.  Forecasting financial time series volatility using Particle Swarm Optimization trained Quantile Regression Neural Network , 2017, Appl. Soft Comput..

[22]  Esmaeil S. Nadimi,et al.  Bayesian state prediction of wind turbine bearing failure , 2018 .

[23]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[24]  Hitoshi Iba,et al.  Particle swarm optimization with Gaussian mutation , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[25]  Antonio J. Rivera,et al.  Dealing with seasonality by narrowing the training set in time series forecasting with kNN , 2018, Expert Syst. Appl..

[26]  Jie Yu,et al.  A Bayesian model averaging based multi-kernel Gaussian process regression framework for nonlinear state estimation and quality prediction of multiphase batch processes with transient dynamics and uncertainty , 2013 .

[27]  Pritpal Singh,et al.  Forecasting stock index price based on M-factors fuzzy time series and particle swarm optimization , 2014, Int. J. Approx. Reason..

[28]  Subrata Bhowmik,et al.  Performance-exhaust emission prediction of diesosenol fueled diesel engine: An ANN coupled MORSM based optimization , 2018, Energy.

[29]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[30]  D. Titterington.  Recursive Parameter Estimation Using Incomplete Data , 1984 .

[31]  Xiangguang Chen,et al.  Soft sensor development for online quality prediction of industrial batch rubber mixing process using ensemble just-in-time Gaussian process regression models , 2016 .

[32]  Carl E. Rasmussen,et al.  Gaussian Processes for Machine Learning (GPML) Toolbox , 2010, J. Mach. Learn. Res..

[33]  Luca Scrucca,et al.  Identifying connected components in Gaussian finite mixture models for clustering , 2016, Comput. Stat. Data Anal..

[34]  Alexander Y. Sun,et al.  Monthly streamflow forecasting using Gaussian Process Regression , 2014 .

[35]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[36]  Dražen Slišković,et al.  Adaptive soft sensor for online prediction and process monitoring based on a mixture of Gaussian process models , 2013, Comput. Chem. Eng..

[37]  Ryutaro Tanaka,et al.  Effect of different features to drill-wear prediction with back propagation neural network , 2014 .

[38]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[39]  Li Wang,et al.  Adaptive Soft Sensor Development Based on Online Ensemble Gaussian Process Regression for Nonlinear Time-Varying Batch Processes , 2015 .

[40]  Nizar Bouguila,et al.  Simultaneous high-dimensional clustering and feature selection using asymmetric Gaussian mixture models , 2015, Image Vis. Comput..

[41]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[42]  O. Cappé,et al.  On‐line expectation–maximization algorithm for latent data models , 2009 .

[43]  J. Röhmel.  The permutation distribution of the Friedman test , 1997 .

[44]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[45]  Wei Zhong,et al.  Short-term Electric Load Forecasting Based on Wavelet Neural Network, Particle Swarm Optimization and Ensemble Empirical Mode Decomposition , 2017 .

[46]  Gordon Lightbody,et al.  Gaussian process approach for modelling of nonlinear systems , 2009, Eng. Appl. Artif. Intell..

[47]  Mukul Agarwal,et al.  Prediction of infrequently measurable quantities in poorly modelled processes , 1995 .