Evaluating Kohonen's learning rule: An approach through genetic algorithms

This paper examines the technical foundations of the self-organising map (SOM). It compares Kohonen’s heuristic-based training algorithm with direct optimisation of a locally-weighted distortion index, also used by Kohonen. Direct optimisation is achieved through a genetic algorithm (GA). Although GAs have been used before with the SOM, this has not been done in conjunction with the distortion index. Comparing heuristic-based training and direct optimisation for the SOM is analogous to comparing the Backpropagation algorithm for feedforward networks with direct optimisation of RMS error. Our experiments reveal lower values of the distortion index with direct optimisation. As to whether the heuristic-based algorithm is able to provide an approximation to gradient descent, our results suggest the answer should be in the negative. Theorems for one-dimensional and for square maps indicate that different point densities will emerge for the two training approaches. Our findings are in accordance with these results.

[1]  Teuvo Kohonen,et al.  Comparison of SOM Point Densities Based on Different Criteria , 1999, Neural Computation.

[2]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[3]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[4]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[5]  H. White.  Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models , 1989 .

[6]  Randall S. Sexton,et al.  Comparing backpropagation with a genetic algorithm for neural network training , 1999 .

[7]  Alfred Inselberg,et al.  Parallel coordinates: a tool for visualizing multi-dimensional geometry , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[8]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[9]  B. S. Everitt,et al.  Cluster analysis , 2014, Encyclopedia of Social Network Analysis and Mining.

[10]  Terrence J. Sejnowski,et al.  A Unifying Objective Function for Topographic Mappings , 1997, Neural Computation.

[11]  Gilles Pagès,et al.  Two or three things that we know about the Kohonen algorithm , 1994, ESANN.

[12]  Daniel Polani.  On the Optimization of Self-Organizing Maps by Genetic Algorithms , 1999 .

[13]  Atsuyuki Okabe,et al.  Spatial Tessellations: Concepts and Applications of Voronoi Diagrams , 1992, Wiley Series in Probability and Mathematical Statistics.

[14]  Paul Phillips,et al.  The Kohonen self‐organizing map: an application to the study of strategic groups in the UK hotel industry , 2001, Expert Syst. J. Knowl. Eng..

[15]  David West,et al.  A comparison of SOM neural network and hierarchical clustering methods , 1996 .

[16]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[17]  Erkki Oja,et al.  Kohonen Maps , 1999, Encyclopedia of Machine Learning.

[18]  K. Schulten,et al.  On the stationary state of Kohonen's self-organizing sensory mapping , 2004, Biological Cybernetics.

[19]  K. Schulten,et al.  Kohonen's self-organizing maps: exploring their computational capabilities , 1988, IEEE 1988 International Conference on Neural Networks.

[20]  Brian Everitt,et al.  Cluster analysis , 1974 .

[21]  Ming S. Hung,et al.  Training neural networks with the GRG2 nonlinear optimizer , 1993 .

[22]  Helge Ritter.  Asymptotic level density for a class of vector quantization processes , 1991, IEEE Trans. Neural Networks.

[23]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[24]  John A. Flanagan,et al.  Self-organisation in Kohonen's SOM , 1996, Neural Networks.

[25]  Van Hulle MM.  Kernel-Based Equiprobabilistic Topographic Map Formation. , 1998, Neural computation.

[26]  T. Sejnowski,et al.  A unifying measure for neighbourhood preservation in topographic mappings , 1997 .

[27]  Niels G. Waller,et al.  A comparison of the classification capabilities of the 1-dimensional kohonen neural network with two pratitioning and three hierarchical cluster analysis algorithms , 1998 .

[28]  T. Samad,et al.  Genetic optimization of self-organizing feature maps , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[29]  Vladimir Cherkassky,et al.  Self-Organization as an Iterative Kernel Smoothing Process , 1995, Neural Computation.

[30]  Klaus Schulten,et al.  Self-organizing maps: ordering, convergence properties and energy functions , 1992, Biological Cybernetics.

[31]  Wolfgang Härdle,et al.  Applied Nonparametric Regression: The kernel method , 1990 .

[32]  B. Curry,et al.  Neural networks: a need for caution , 1997 .

[33]  W. Härdle.  Applied Nonparametric Regression , 1991 .

[34]  Stephen P. Luttrell,et al.  Derivation of a class of training algorithms , 1990, IEEE Trans. Neural Networks.

[35]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .