Optimization of self-organizing polynomial neural networks

The main disadvantage of self-organizing polynomial neural networks (SOPNN) automatically structured and trained by the group method of data handling (GMDH) algorithm is a partial optimization of model weights as the GMDH algorithm optimizes only the weights of the topmost (output) node. In order to estimate to what extent the approximation accuracy of the obtained model can be improved the particle swarm optimization (PSO) has been used for the optimization of weights of all node-polynomials. Since the PSO is generally computationally expensive and time consuming a more efficient Levenberg-Marquardt (LM) algorithm is adapted for the optimization of the SOPNN. After it has been optimized by the LM algorithm the SOPNN outperformed the corresponding models based on artificial neural networks (ANN) and support vector method (SVM). The research is based on the meta-modeling of the thermodynamic effects in fluid flow measurements with time-constraints. The outstanding characteristics of the optimized SOPNN models are also demonstrated in learning the recurrence relations of multiple superimposed oscillations (MSO).

[1]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[2]  Jürgen Schmidhuber,et al.  Training Recurrent Networks by Evolino , 2007, Neural Computation.

[3]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[4]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[5]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[6]  T. Simpson,et al.  Comparative studies of metamodeling techniques under multiple modeling criteria , 2000 .

[7]  Ivan Maric,et al.  Self-Organizing Polynomial Networks for Time-Constrained Applications , 2011, IEEE Transactions on Industrial Electronics.

[8]  Georges G. E. Gielen,et al.  Recurrent sparse support vector regression machines trained by active learning in the time-domain , 2012, Expert Syst. Appl..

[9]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[10]  Ivan Ivek,et al.  Compensation for Joule–Thomson effect in flowrate measurements by GMDH polynomial , 2010 .

[11]  Steven C. Chapra,et al.  Numerical Methods for Engineers , 1986 .

[12]  Robert F. Stengel,et al.  Smooth function approximation using neural networks , 2005, IEEE Transactions on Neural Networks.

[13]  Nobuyuki Matsui,et al.  GMDH-based modeling and feedforward compensation for nonlinear friction in table drive systems , 2003, IEEE Trans. Ind. Electron..

[14]  Simon Haykin,et al.  Decoupled echo state networks with lateral inhibition , 2007, Neural Networks.

[15]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[16]  Hitoshi Iba,et al.  Learning polynomial feedforward neural networks by genetic programming and backpropagation , 2003, IEEE Trans. Neural Networks.

[17]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[18]  Helmut Hauser,et al.  Echo state networks with filter neurons and a delay&sum readout , 2010, Neural Networks.