Gaussian Process Models for Systems Identification

Different models can be used for nonlinear dy- namic systems identification and the Gaussian process model is a relatively new option with several interesting features: model predictions contain the measure of confidence, the model has a small number of training parameters and facilitated structure determination, and different possibilities of including prior knowledge exist. In this paper the framework for the identification of a dynamic system model based on the Gaussian processes is presented and a short survey with a comprehensive bibliography of published works on application of Gaussian processes for modelling of dynamic systems is given. I. INTRODUCTION While there are numerous methods for the identification of linear dynamic systems from measured data, the non- linear systems identification requires more sophisticated ap- proaches. The most common choices include artificial neural networks, fuzzy models and others. Gaussian process (GP) models present a new, emerging, complementary method for nonlinear system identification. The GP model is a probabilistic, non-parametric black- box model. It differs from most of the other black-box identification approaches as it does not try to approximate the modelled system by fitting the parameters of the selected basis functions but rather searches for the relationship among measured data. Gaussian process models are closely related to approaches such as Support Vector Machines and specially Relevance Vector Machines (3). The output of the Gaussian process model is a normal distribution, expressed in terms of mean and variance. The mean value represents the most likely output and the vari- ance can be interpreted as the measure of its confidence. The obtained variance, which depends on the amount and quality of available identification data, is important infor- mation distinguishing the GP models from other methods. The GP model structure determination is facilitated as only the covariance function and the regressors of the model need to be selected. Another potentially useful attribute of the GP model is the possibility to include various kinds of prior knowledge into the model, see e.g. (46) for the incorporation of local models and the static characteristic. Also the number of model parameters, which need to be optimised is smaller than in other black-box identification approaches. The disadvantage of the method is the potential computational burden for optimization that increases with amount of data and number of regressors. The GP model was first used for solving a regression problem in the late seventies, but it gained popularity within the machine learning community in the late nineties of the twentieth century. Results of a possible implementation of the GP model for the identification of dynamic systems were presented only recently, e.g. (11), (54). The investigation of the model with uncertain inputs, which enables the propaga- tion of uncertainty through the model, is given in (20), (33), (39) and illustrated in (27), (47) and many others. The purpose of this paper is twofold. First, to present the procedure of dynamic system identification using the model based on Gaussian processes taken from (83). Second, a comprehensive bibliography of published works on Gaussian processes application for modelling of dynamic systems with a short survey is given. Many of dynamic systems are often considered as com- plex, however simplified input/output behaviour representa- tions are sufficient for certain purposes, e.g. feedback control design, prediction models for supervisory control, etc. In the paper it is explained how the advantages of Gaussian process models can be used in identification and validation of such models.