Heuristic and optimization approaches to extending the Kohonen self organizing algorithm

Abstract The Kohonen self organizing neural network has been applied to an increasingly wider range of application problems that traditionally have been the domain of statistical and operational research techniques, such as data clustering and classification, and optimization and control. This Kohonen network is bestowed with a number of unique strengths which are, unfortunately, matched by an equally formidable set of limitations due its learning algorithm. There have been extensive studies over the last decade to extend the Kohonen neural network using heuristic and optimization approaches. This paper provides a comprehensive survey of the research efforts directed to enhancing the Kohonen self organizing neural network and its learning algorithm. We also point out some research directions for pursuing further improvements.

[1]  Thomas P. Yunck,et al.  A Technique to Identify Nearest Neighbors , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  András Faragó,et al.  Fast Nearest-Neighbor Search in Dissimilarity Spaces , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Y. Cheng,et al.  Clustering with competing self-organizing maps , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[4]  V. V. Tolat,et al.  A self-organizing neural network for classifying sequences , 1989, International 1989 Joint Conference on Neural Networks.

[5]  Sooyoung Kim,et al.  Neural network applications for scheduling jobs on parallel machines , 1993 .

[6]  Bernard Angéniol,et al.  Self-organizing feature maps and the travelling salesman problem , 1988, Neural Networks.

[7]  Cheng-Chang Lu,et al.  Neural networks for classified vector quantization of images , 1992, Electronic Imaging.

[8]  B. Fritzke,et al.  FLEXMAP-a neural network for the traveling salesman problem with linear time and space complexity , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[9]  J. Lampinen,et al.  Fast self-organization by the probing algorithm , 1989, International 1989 Joint Conference on Neural Networks.

[10]  Teuvo Kohonen,et al.  Improved versions of learning vector quantization , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[11]  Bernd Fritzke,et al.  Unsupervised clustering with growing cell structures , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[12]  O. G. Jakubowicz.  Multi-layer multi-feature map architecture for situational analysis , 1989, International 1989 Joint Conference on Neural Networks.

[13]  Stephen Grossberg,et al.  A self-organizing neural network model for redundant sensory-motor control, motor equivalence, and tool use , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[14]  T. Wada,et al.  Optimization by extended LVQ , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[15]  Yasuo Matsuyama,et al.  Self-organization via competition, cooperation and categorization applied to extended vehicle routing problems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[16]  Jorma Laaksonen,et al.  SOM_PAK: The Self-Organizing Map Program Package , 1996 .

[17]  John G. Taylor,et al.  The temporal Kohönen map , 1993, Neural Networks.

[18]  Jesús Cid-Sueiro,et al.  Self-Organizing Feature Maps and Their Application to Digital Coding of Information , 1991, IWANN.

[19]  Zhen-Ping Lo,et al.  Analysis of the convergence properties of topology preserving neural networks , 1993, IEEE Trans. Neural Networks.

[20]  Jari Kangas,et al.  Time-delayed self-organizing maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[21]  Mohamed A. Nour.  Improved clustering and classification algorithms for the Kohonen self-organizing neural network , 1994 .

[22]  Klaus Schulten,et al.  Implementation of self-organizing neural networks for visuo-motor control of an industrial robot , 1993, IEEE Trans. Neural Networks.

[23]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[24]  F. Palmieri.  A self-organizing neural network for multidimensional approximation , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[25]  Suchendra M. Bhandarkar,et al.  A multi-layer Kohonen's self-organizing feature map for range image segmentation , 1993, IEEE International Conference on Neural Networks.

[26]  S. Lu,et al.  Texture segmentation by clustering of Gabor feature vectors , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[27]  Terrance L. Huntsberger,et al.  PARALLEL SELF-ORGANIZING FEATURE MAPS FOR UNSUPERVISED PATTERN RECOGNITION , 1990 .

[28]  Y. Idan,et al.  Handwritten digits recognition by a supervised Kohonen-like learning algorithm , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[29]  Keinosuke Fukunaga,et al.  A Branch and Bound Algorithm for Computing k-Nearest Neighbors , 1975, IEEE Transactions on Computers.

[30]  Sigeru Omatu,et al.  Pattern Classification for Remote Sensing Data Using Neural Network , 1991 .

[31]  Stanley C. Ahalt,et al.  Competitive learning algorithms for vector quantization , 1990, Neural Networks.

[32]  S. Danielson.  Recognition of Danish phonemes using an artificial neural network , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[33]  Constantin V. Negoita,et al.  On Fuzzy Systems , 1978 .

[34]  A. Nyeck,et al.  An improved fast adaptive search algorithm for vector quantization by progressive codebook arrangement , 1992, Pattern Recognit..

[35]  Christian Jutten,et al.  Adaptive Optimization of Neural Algorithms , 1991, IWANN.

[36]  Marimuthu Palaniswami,et al.  An extended self-organizing map with gated neurons , 1993, IEEE International Conference on Neural Networks.

[37]  V. Cherkassky,et al.  Self-organizing network for regression: efficient implementation and comparative evaluation , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[38]  Karl Goser,et al.  A Modified Algorithm for Self-Organizing Maps Based on the Schrödinger Equation , 1991, IWANN.

[39]  Yasuo Matsuyama,et al.  Self-organizing neural networks and various euclidean traveling salesman problems , 1992, Systems and Computers in Japan.

[40]  Yasuo Matsuyama,et al.  Competitive self-organization and combinatorial optimization: applications to traveling salesman problem , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[41]  Harald Hruschka,et al.  Determining market response functions by neural network modeling: A comparison to econometric techniques , 1993 .

[42]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[43]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[44]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[45]  Kunihiko Fukushima,et al.  Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position , 1982, Pattern Recognit..

[46]  Behrooz Kamgar-Parsi,et al.  An improved branch and bound algorithm for computing k-nearest neighbors , 1985, Pattern Recognit. Lett..

[47]  S. Lu.  Pattern classification using self-organizing feature maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[48]  Jorma Laaksonen,et al.  Variants of self-organizing maps , 1990, International 1989 Joint Conference on Neural Networks.

[49]  Ray-I Chang,et al.  Force directed self-organizing map and its application to VLSI cell placement , 1993, IEEE International Conference on Neural Networks.

[50]  Shun-ichi Amari,et al.  Field theory of self-organizing neural nets , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[51]  Shouhong Wang,et al.  Neural network techniques for monotonic nonlinear models , 1994, Comput. Oper. Res..

[52]  T. Samad,et al.  Genetic optimization of self-organizing feature maps , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[53]  G.R. De Haan,et al.  Links between self-organizing feature maps and weighted vector quantization , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[54]  Jörg A. Walter,et al.  Nonlinear prediction with self-organizing maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[55]  Teuvo Kohonen,et al.  Things you haven't heard about the self-organizing map , 1993, IEEE International Conference on Neural Networks.

[56]  G. J. Hueter.  Solution of the traveling salesman problem with an adaptive ring , 1988, IEEE 1988 International Conference on Neural Networks.

[57]  Utpal Roy,et al.  Connectionist models for part-family classifications , 1993 .

[58]  Ahmed Hemani,et al.  Scheduling by Self-organization , 1990 .

[59]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[60]  Pasi Koikkalainen,et al.  Self-organizing hierarchical feature maps , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[61]  X. Xu,et al.  Effective neural algorithms for the traveling salesman problem , 1991, Neural Networks.

[62]  Wen-Tsuen Chen,et al.  A neural network model which combines unsupervised and supervised learning , 1993, IEEE Trans. Neural Networks.

[63]  Nallan C. Suresh,et al.  An improved neural network leader algorithm for part-machine grouping in group technology , 1993 .

[64]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[65]  H. Ritter.  Combining self-organizing maps , 1989, International 1989 Joint Conference on Neural Networks.

[66]  James C. Bezdek,et al.  Generalized clustering networks and Kohonen's self-organizing scheme , 1993, IEEE Trans. Neural Networks.

[67]  Z.-P. Lo,et al.  Two theorems for the Kohonen mapping neural network , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[68]  James C. Bezdek,et al.  Fuzzy Kohonen clustering networks , 1992, [1992 Proceedings] IEEE International Conference on Fuzzy Systems.

[69]  Roberto Togneri,et al.  Parallel implementation of the Kohonen algorithm on transputer , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[70]  R. M. Holdaway.  Enhancing supervised learning algorithms via self-organization , 1989, International 1989 Joint Conference on Neural Networks.

[71]  Koji Tochinai,et al.  An adaptive recognition using self-organized network , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[72]  Ralph Linsker,et al.  How to Generate Ordered Maps by Maximizing the Mutual Information between Input and Output Signals , 1989, Neural Computation.

[73]  David M. Skapura,et al.  Neural networks - algorithms, applications, and programming techniques , 1991, Computation and neural systems series.

[74]  Forest Baskett,et al.  An Algorithm for Finding Nearest Neighbors , 1975, IEEE Transactions on Computers.

[75]  D. A. Mlynski,et al.  Neural somatotopical mapping for VLSI placement optimization , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[76]  Ralph Linsker,et al.  Towards an Organizing Principle for a Layered Perceptual Network , 1987, NIPS.

[77]  Jon Louis Bentley,et al.  An Algorithm for Finding Best Matches in Logarithmic Expected Time , 1977, TOMS.

[78]  Robert M. Gray,et al.  An Improvement of the Minimum Distortion Encoding Algorithm for Vector Quantization , 1985, IEEE Trans. Commun..