An Empirical Evaluation of Robust Gaussian Process Models for System Identification

System identification comprises a number of linear and nonlinear tools for black-box modeling of dynamical systems, with applications in several areas of engineering, control, biology and economy. However, the usual Gaussian noise assumption is not always satisfied, specially if data is corrupted by impulsive noise or outliers. Bearing this in mind, the present paper aims at evaluating how Gaussian Process (GP) models perform in system identification tasks in the presence of outliers. More specifically, we compare the performances of two existing robust GP-based regression models in experiments involving five benchmarking datasets with controlled outlier inclusion. The results indicate that, although still sensitive in some degree to the presence of outliers, the robust models are indeed able to achieve lower prediction errors in corrupted scenarios when compared to conventional GP-based approach.

[1]  Carl E. Rasmussen,et al.  Derivative Observations in Gaussian Process Models of Dynamic Systems , 2002, NIPS.

[2]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[3]  Malte Kuss,et al.  Approximate inference for robust Gaussian process regression , 2005 .

[4]  Neil D. Lawrence,et al.  Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data , 2003, NIPS.

[5]  Malte Kuß,et al.  Gaussian process models for robust regression, classification, and reinforcement learning , 2006 .

[6]  Aki Vehtari,et al.  Robust Gaussian Process Regression with a Student-t Likelihood , 2011, J. Mach. Learn. Res..

[7]  Jus Kocijan,et al.  Dynamical systems identification using Gaussian process models with incorporated local models , 2011, Eng. Appl. Artif. Intell..

[8]  T. Johansen,et al.  On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures , 1999, 1999 European Control Conference (ECC).

[9]  Hirotaka Nakayama,et al.  A Computational Intelligence Approach to Optimization with Unknown Objective Functions , 2001, ICANN.

[10]  Carl E. Rasmussen,et al.  Variational Gaussian Process State-Space Models , 2014, NIPS.

[11]  Ganapati Panda,et al.  Robust identification of nonlinear complex systems using low complexity ANN and particle swarm optimization technique , 2011, Expert Syst. Appl..

[12]  Agathe Girard,et al.  Dynamic systems identification with Gaussian processes , 2005 .

[13]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[14]  Benjamin Berger,et al.  Robust Gaussian Process Modelling for Engine Calibration , 2012 .

[15]  David Barber,et al.  Bayesian Classification With Gaussian Processes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Neil D. Lawrence,et al.  Variational inference for Student-t models: Robust Bayesian interpolation and generalised component analysis , 2005, Neurocomputing.

[17]  Geoffrey E. Hinton,et al.  Evaluation of Gaussian processes and other methods for non-linear regression , 1997 .

[18]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[19]  Wolfram Burgard,et al.  Learning Non-stationary System Dynamics Online Using Gaussian Processes , 2010, DAGM-Symposium.

[20]  Jus Kocijan,et al.  Evolving Gaussian process models for prediction of ozone concentration in the air , 2013, Simul. Model. Pract. Theory.