Efficient learning of NN-MLP based on individual evolutionary algorithm

Abstract The nearest neighbor based multilayer perception (NN-MLP) is a suitable model for self-organization, and has been studied by many authors in different forms. However, a large number of neurons are usually required in this kind of networks. To obtain smaller or the smallest NN-MLP, this paper introduces the concept of individual evolutionary algorithm (IEA), and proposes a new method for NN-MLP learning. There are four basic operations in the IEA: competition, gain, loss and retraining. The basic rule is: all individuals compete for surviving, winners gain more, losers lose more, and the individuals are retrained to function better than before. The learning algorithm based on the IEA is simple and suitable for parallel realization, and is able to produce the ‘smallest-at-present’ networks from random ones in an evolutionary manner. Its efficiency is shown by experimental results.

[1]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[2]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[3]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[4]  O. J. Murphy,et al.  Nearest neighbor pattern classification perceptrons , 1990, Proc. IEEE.

[5]  Tsu-Shuan Chang,et al.  A universal neural net with guaranteed convergence to zero system error , 1992, IEEE Trans. Signal Process..

[6]  Peter E. Hart,et al.  The condensed nearest neighbor rule (Corresp.) , 1968, IEEE Trans. Inf. Theory.

[7]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[8]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[9]  Richard P. Brent,et al.  Fast training algorithms for multilayer neural nets , 1991, IEEE Trans. Neural Networks.

[10]  Saul B. Gelfand,et al.  Classification trees with neural network feature extraction , 1992, IEEE Trans. Neural Networks.

[11]  Nirmal K. Bose,et al.  Neural network design using Voronoi diagrams , 1993, IEEE Trans. Neural Networks.

[12]  Chin-Liang Chang,et al.  Finding Prototypes For Nearest Neighbor Classifiers , 1974, IEEE Transactions on Computers.

[13]  Keisuke Kameyama,et al.  Neural network pruning by fusing hidden layer units , 1991 .

[14]  Bart Kosko,et al.  Stochastic competitive learning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[15]  Shlomo Geva,et al.  Adaptive nearest neighbor pattern classification , 1991, IEEE Trans. Neural Networks.

[16]  Rabab Kreidieh Ward,et al.  Vector Quantization Technique for Nonparametric Classifier Design , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  G. Gates,et al.  The reduced nearest neighbor rule (Corresp.) , 1972, IEEE Trans. Inf. Theory.

[18]  Wayne Ieee,et al.  Entropy Nets: From Decision Trees to Neural Networks , 1990 .

[19]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[20]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[21]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[22]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[23]  Tatsuo Higuchi,et al.  Supervised Organization of Nearest Neighbor MLP , 1994 .

[24]  Keinosuke Fukunaga,et al.  The Reduced Parzen Classifier , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Xin Yao,et al.  Evolutionary Artificial Neural Networks , 1993, Int. J. Neural Syst..

[26]  Bart Kosko,et al.  Unsupervised learning in noise , 1990, International 1989 Joint Conference on Neural Networks.

[27]  Jorma Laaksonen,et al.  Variants of self-organizing maps , 1990, International 1989 Joint Conference on Neural Networks.

[28]  Jocelyn Sietsma,et al.  Creating artificial neural networks that generalize , 1991, Neural Networks.

[29]  Qiangfu Zhao.  Neural Network Realization of the Nearest Neighbor Method , 1993 .

[30]  K. Fukunaga,et al.  Nonparametric Data Reduction , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.