Local Model Network Identification With Gaussian Processes

A Bayesian Gaussian process (GP) modeling approach has recently been introduced to model-based control strategies. The estimate of the variance of the predicted output is the most useful advantage of GPs in comparison to neural networks (NNs) and fuzzy models. However, the GP model is computationally demanding and nontransparent. To reduce the computation load and increase transparency, a local linear GP model network is proposed in this paper. The proposed methodology combines the local model network principle with the GP prior approach. A novel algorithm for structure determination and optimization is introduced, which is widely applicable to the training of local model networks. The modeling procedure of the local linear GP (LGP) model network is demonstrated on an example of a nonlinear laboratory scale process rig.

[1]  Kumpati S. Narendra,et al.  Adaptive control using multiple models , 1997, IEEE Trans. Autom. Control..

[2]  Jonas Sjöberg,et al.  Gaussian processes framework for validation of linear and nonlinear models , 2003 .

[3]  W. Leithead,et al.  Analytic framework for blended multiple model systems using linear local models , 1999 .

[4]  Gordon Lightbody,et al.  Gaussian process approaches to nonlinear modelling for control , 2005 .

[5]  Alan F. Murray,et al.  Confidence estimation methods for neural networks : a practical comparison , 2001, ESANN.

[6]  Roderick Murray-Smith,et al.  A local model network approach to nonlinear modelling , 1994 .

[7]  Matthias W. Seeger,et al.  Bayesian Gaussian process models : PAC-Bayesian generalisation error bounds and sparse approximations , 2003 .

[8]  Carl E. Rasmussen,et al.  In Advances in Neural Information Processing Systems , 2011 .

[9]  Iain Murray.  Introduction To Gaussian Processes , 2008 .

[10]  Tony J. Dodd,et al.  Identification of non-linear time series via kernels , 2002, Int. J. Syst. Sci..

[11]  Wallace E. Larimore,et al.  Optimal Reduced Rank Modeling, Prediction, Monitoring and Control using Canonical Variate Analysis , 1997 .

[12]  Kumpati S. Narendra,et al.  Adaptation and learning using multiple models, switching, and tuning , 1995 .

[13]  Roderick Murray-Smith,et al.  Multiple Model Approaches to Modelling and Control , 1997 .

[14]  Christopher M. Bishop,et al.  Variational Relevance Vector Machines , 2000, UAI.

[15]  Steve R. Waterhouse,et al.  Bayesian Methods for Mixtures of Experts , 1995, NIPS.

[16]  Agathe Girard,et al.  Dynamic systems identification with Gaussian processes , 2005 .

[17]  George W. Irwin,et al.  Local model networks for nonlinear system identification , 1997 .

[18]  Roderick Murray-Smith,et al.  Divide & conquer identification using Gaussian process priors , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[19]  J. Kocijan,et al.  Predictive control with Gaussian process models , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[20]  Tor Arne Johansen,et al.  Speed control design for an experimental vehicle using a generalized gain scheduling approach , 2000, IEEE Trans. Control. Syst. Technol..

[21]  David Barber,et al.  Bayesian Classification With Gaussian Processes , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Roderick Murray-Smith,et al.  Gaussian process priors with ARMA noise models , 2001 .

[23]  William P. Marnane,et al.  Gaussian Process Modelling as an Indicator of Neonatal Seizure , 2006, SPPRA.

[24]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[25]  Yunqian Ma,et al.  Multiple model regression estimation , 2005, IEEE Transactions on Neural Networks.

[26]  A. O'Hagan,et al.  Curve Fitting and Optimal Design for Prediction , 1978 .

[27]  S. Sathiya Keerthi,et al.  Parallel sequential minimal optimization for the training of support vector machines , 2006, IEEE Trans. Neural Networks.

[28]  Gregor Gregorcic,et al.  Internal model control based on a Gaussian process prior model , 2003, Proceedings of the 2003 American Control Conference, 2003..

[29]  J. Príncipe,et al.  Local dynamic modeling with self-organizing maps and applications to nonlinear system identification and control , 1998, Proc. IEEE.

[30]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[31]  T. Johansen,et al.  Constructing NARMAX models using ARMAX models , 1993 .

[32]  Geoffrey E. Hinton,et al.  Evaluation of Gaussian processes and other methods for non-linear regression , 1997 .

[33]  David S. Broomhead,et al.  Delay Embeddings for Forced Systems. II. Stochastic Forcing , 2003, J. Nonlinear Sci..

[34]  F. Flentge,et al.  Locally Weighted Interpolating Growing Neural Gas , 2006, IEEE Transactions on Neural Networks.

[35]  Jeongho Cho,et al.  Modeling and inverse controller design for an unmanned aerial vehicle based on the self-organizing map , 2006, IEEE Transactions on Neural Networks.

[36]  Lyle H. Ungar,et al.  Using radial basis functions to approximate a function and its error bounds , 1992, IEEE Trans. Neural Networks.

[37]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[38]  Helge J. Ritter,et al.  Self-Organizing Feature Maps for Modeling and Control of Robotic Manipulators , 2003, J. Intell. Robotic Syst..

[39]  Marimuthu Palaniswami,et al.  Incremental training of support vector machines , 2005, IEEE Transactions on Neural Networks.

[40]  T. Johansen,et al.  On transient dynamics, off-equilibrium behaviour and identification in blended multiple model structures , 1999, 1999 European Control Conference (ECC).

[41]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[42]  Anthony Zaknich,et al.  A practical sub-space adaptive filter , 2003, Neural Networks.

[43]  Christopher K. I. Williams.  Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[44]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[45]  G. Irwin,et al.  Nonlinear internal model control using local model networks , 1997 .

[46]  David J. C. Mackay,et al.  Introduction to Monte Carlo Methods , 1998, Learning in Graphical Models.

[47]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[48]  D. M. Titterington,et al.  Bayesian regression and classification using mixtures of Gaussian processes , 2003 .

[49]  Robert Shorten,et al.  On the interpretation of local models in blended multiple model structures. , 1999 .

[50]  Tor Arne Johansen,et al.  Identification of non-linear system structure and parameters using regime decomposition , 1995, Autom..

[51]  Yoram Reich,et al.  Evaluating machine learning models for engineering problems , 1999, Artif. Intell. Eng..