Identification of MIMO systems using MLP networks: Comparison between SVR and random initialisation

Neural network (NN) modelling approach is often used for non-linear system identification. Building a NN for some identification problem starts by choosing its structure and initial weights. There is no exact method to determine the optimal initialisation for a NN, but some authors have used support vector regression (SVR) to initialise a RBFNN which could be considered as a systematic way. This paper presents a SVR initialisation method for Multi-Layer Perceptron (MLP) NN. The proposed method is based on the analogy between NN and SVR to determine the necessary number of hidden neurons and the initial weights for a given modelling precision. Simulation results for multi-input multi-output (MIMO) system show the feasibility and accuracy of the proposed method.

[1]  Jin Zhang,et al.  Adaptive neural network control for a class of MIMO nonlinear systems with disturbances in discrete-time , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Wencong Lu,et al.  Using support vector regression for the prediction of the band gap and melting point of binary and ternary compound semiconductors , 2006 .

[3]  Chia-Ju Wu,et al.  Identification of MIMO systems using radial basis function networks with hybrid learning algorithm , 2009, Appl. Math. Comput..

[4]  Sven F. Crone,et al.  Parameter Sensitivity of Support Vector Regression and Neural Networks for Forecasting , 2006, DMIN.

[5]  B. Achiriloaie,et al.  VI REFERENCES , 1961 .

[6]  L. Thiaw.  Identification de systèmes dynamiques non linéaires par réseaux de neurones et multimodèles , 2008 .

[7]  Tommy W. S. Chow,et al.  A weight initialization method for improving training speed in feedforward neural network , 2000, Neurocomputing.

[8]  Hsuan-Tien Lin.  A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .

[9]  Nabil Benoudjit,et al.  Multiple architecture system for wind speed prediction , 2011 .

[10]  Mercedes Fernández-Redondo,et al.  Weight initialization methods for multilayer feedforward , 2001, ESANN.

[11]  Mario Innocenti,et al.  Neural Networks and other Techniques for Fault Identification and Isolation of Aircraft Systems , 2003 .

[12]  Artificial Neural Network Technique for Forecasting : Neural Tree and Necessary Number of Hidden Units , 2009 .

[13]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[14]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[15]  Stephan Trenn,et al.  Multilayer Perceptrons: Approximation Order and Necessary Number of Hidden Units , 2008, IEEE Transactions on Neural Networks.

[16]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.