Hybrid kernel approach to Gaussian process modeling with colored noises

Abstract Traditional Gaussian Process Regression (TGPR) models typically assume independent identically distributed (i.i.d.) noises for all observations. However, applications with colored noises/disturbances frequently arise in modeling of complex processes. In this work, first, we consider to model noise by an ARMA time series model and determine its covariance for subsequent Gaussian Process (GP) modeling. Then a novel approach based on hybrid kernels is proposed, thereby avoiding parametric modeling of the colored noises. Moreover, all hyper-parameters are estimated simultaneously by using a particle swarm optimization (PSO) algorithm. Finally, a synthetic data, a simulated example, as well as a polyester polymerization process are used to demonstrate the effectiveness of the proposed approach.

[1]  Idelfonso B. R. Nogueira,et al.  Optimization strategies for chiral separation by true moving bed chromatography using Particles Swarm Optimization (PSO) and new Parallel PSO variant , 2019, Comput. Chem. Eng..

[2]  Malte Kuß,et al.  Gaussian process models for robust regression, classification, and reinforcement learning , 2006 .

[3]  Weiming Shao,et al.  Nonlinear industrial soft sensor development based on semi-supervised probabilistic mixture of extreme learning machines , 2019, Control Engineering Practice.

[4]  Biao Huang,et al.  Gaussian process modelling with Gaussian mixture likelihood , 2019, Journal of Process Control.

[5]  Roderick Murray-Smith,et al.  Gaussian process priors with ARMA noise models , 2001 .

[6]  Yongsheng Ding,et al.  PET Viscosity Prediction Using JIT-based Extreme Learning Machine , 2018 .

[7]  Dong Hwi Jeong,et al.  Enhancement of modifier adaptation scheme via feedforward decision maker using historical disturbance data and deep machine learning , 2018, Comput. Chem. Eng..

[8]  Biao Huang,et al.  Identification of robust Gaussian Process Regression with noisy input using EM algorithm , 2019 .

[9]  Manabu Kano,et al.  Locally weighted kernel partial least squares regression based on sparse nonlinear features for virtual sensing of nonlinear time-varying processes , 2017, Comput. Chem. Eng..

[10]  Biao Huang,et al.  Multivariate Gaussian process regression for nonlinear modelling with colored noise , 2018, Trans. Inst. Meas. Control.

[11]  Andreas Svensson,et al.  Probabilistic forecasting of electricity consumption, photovoltaic power generation and net demand of an individual building using Gaussian Processes , 2018 .

[12]  Wei Zhang,et al.  JITL based MWGPR soft sensor for multi-mode process with dual-updating strategy , 2016, Comput. Chem. Eng..

[13]  J. Leeuw.  The covariance matrix of ARMA errors in closed form , 1994 .

[14]  Biao Huang,et al.  Robust Gaussian process modeling using EM algorithm , 2016 .

[15]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[16]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[17]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[18]  Roman Rosipal,et al.  Kernel Partial Least Squares Regression in Reproducing Kernel Hilbert Space , 2002, J. Mach. Learn. Res..

[19]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[20]  David Duvenaud,et al.  Automatic model construction with Gaussian processes , 2014 .

[21]  B. Kowalski,et al.  Partial least-squares regression: a tutorial , 1986 .

[22]  Jie Yu,et al.  Online quality prediction of nonlinear and non-Gaussian chemical processes with shifting dynamics using finite mixture model based Gaussian process regression approach , 2012 .

[23]  Soon Keat Tan,et al.  Moving-Window GPR for Nonlinear Dynamic System Modeling with Dual Updating and Dual Preprocessing , 2012 .

[24]  Alexander J. Smola,et al.  Heteroscedastic Gaussian process regression , 2005, ICML.

[25]  Biao Huang,et al.  Identification of nonlinear parameter varying systems with missing output data , 2012 .

[26]  Jaesik Choi,et al.  Discovering Latent Covariance Structures for Multiple Time Series , 2019, ICML.

[27]  Wolfram Burgard,et al.  Most likely heteroscedastic Gaussian process regression , 2007, ICML '07.

[28]  Paul W. Goldberg,et al.  Regression with Input-dependent Noise: A Gaussian Process Treatment , 1997, NIPS.

[29]  Manabu Kano,et al.  A comparative study of deep and shallow predictive techniques for hot metal temperature prediction in blast furnace ironmaking , 2019, Comput. Chem. Eng..

[30]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[31]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .