Self-organizing network for optimum supervised learning

A new algorithm called the self-organizing neural network (SONN) is introduced. Its use is demonstrated in a system identification task. The algorithm constructs a network, chooses the node functions, and adjusts the weights. It is compared to the backpropagation algorithm in the identification of the chaotic time series. The results show that SONN constructs a simpler, more accurate model, requiring less training data and fewer epochs. The algorithm can also be applied as a classifier.

[1]  Dennis Gabor,et al.  A universal nonlinear filter, predictor and simulator which optimizes itself by a learning process , 1961 .

[2]  L. A. Zadeh,et al.  From Circuit Theory to System Theory , 1962, Proceedings of the IRE.

[3]  R. Roy,et al.  A learning technique for Volterra series representation , 1967, IEEE Transactions on Automatic Control.

[4]  Karl Johan Åström,et al.  BOOK REVIEW SYSTEM IDENTIFICATION , 1994, Econometric Theory.

[5]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[6]  H. Akaike.  A new look at the statistical model identification , 1974 .

[7]  Mark A. Franklin,et al.  A Learning Identification Algorithm and Its Application to an Environmental System , 1975, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Saburo Ikeda,et al.  Sequential GMDH Algorithm and Its Application to River Flow Prediction , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[10]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[11]  H. Tamura,et al.  Heuristics free group method of data handling algorithm of generating optimal partial polynomials with application to air pollution prediction , 1980 .

[12]  James P. Crutchfield,et al.  Geometry from a Time Series , 1980 .

[13]  R. Kashyap.  Inconsistency of the AIC rule for estimating the order of autoregressive models , 1980 .

[14]  F. Takens.  Detecting strange attractors in turbulence , 1981 .

[15]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[16]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[17]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Sanjoy Mitter,et al.  Analysis of simulated annealing for optimization , 1985, 1985 24th IEEE Conference on Decision and Control.

[19]  M. Feder.  Maximum entropy as a special case of the minimum description length criterion , 1986, IEEE Trans. Inf. Theory.

[20]  Farmer,et al.  Predicting chaotic time series. , 1987, Physical review letters.

[21]  J. Sanz,et al.  Note on contrast measures and polynomial classifiers , 1988 .