On the modelling of nonlinear dynamic systems using support vector neural networks

Abstract Though neural networks have the ability to approximate nonlinear functions with arbitrary accuracy, good generalization results are obtained only if the structure of the network is suitably chosen. Therefore, selecting the ‘best’ structure of the neural networks is an important problem. Support vector neural networks (SVNN) are proposed in this paper, which can provide a solution to this problem. The structure of the SVNN is obtained by a constrained minimization for a given error bound similar to that in the support vector regression (SVR). After the structure is selected, its weights are computed by the linear least squares method, as it is a linear-in-weight network. Consequently, in contrast to the SVR, the output of the SVNN is unbiased. It is further shown here that the variance of the modelling error of the SVNN is bounded by the square of the given error bound in selecting its structure, and is smaller than that of the SVR. The performance of the SVNN is illustrated by a simulation example involving a benchmark nonlinear system.

[1]  Martin Brown,et al.  SUPANOVA: a sparse, transparent modelling approach , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[2]  Michael Hanss.  Identification of enhanced fuzzy models with special membership functions and fuzzy rule bases , 1999 .

[3]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[4]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[5]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[6]  Bernhard Schölkopf,et al.  Comparing support vector machines with Gaussian kernels to radial basis function classifiers , 1997, IEEE Trans. Signal Process..

[7]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[8]  Martin Brown,et al.  Neurofuzzy adaptive modelling and control , 1994 .

[9]  R. Harrison,et al.  Support Vector Machines for System Identification , 1998 .

[10]  Bernhard Schölkopf,et al.  Shrinking the Tube: A New Support Vector Regression Algorithm , 1998, NIPS.

[11]  P. Caines.  Linear Stochastic Systems , 1988 .

[12]  Stephen A. Billings,et al.  International Journal of Control , 2004 .

[13]  Martin Brown,et al.  Advances in neurofuzzy algorithms for real-time modelling and control , 1996 .

[14]  Yu Wang,et al.  Modelling of nonlinear stochastic dynamical systems using neurofuzzy networks , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[15]  Chris J. Harris,et al.  Neurofuzzy state estimators and their applications , 1999 .

[16]  S. Billings,et al.  Givens rotation based fast backward elimination algorithm for RBF neural network pruning , 1997 .

[17]  Christopher J. C. Burges,et al.  Geometry and invariance in kernel based methods , 1999 .