Self-organizing feature map with a momentum term

Abstract The objectives of this paper are to derive a momentum term in the Kohonen's self-organizing feature map algorithm theoretically and to show the effectiveness of the term by computer simulations. We will derive the self-organizing feature map algorithm having the momentum term through the following assumptions: (1) The cost function is E n = Σ μ n α n − μ E μ , where E μ is the modified Lyapunov function originally proposed by Ritter and Schulten at the μth learning time and α is the momentum coefficient. (2) The latest weights are assumed in calculating the cost function E n . According to our simulations, it has shown that the momentum term in the self-organizing feature map can considerably contribute to the acceleration of the convergence.

[1]  Masafumi Hagiwara.  Novel backpropagation algorithm for reduction of hidden units and acceleration of convergence using artificial selection , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[2]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[3]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[4]  Masafumi Hagiwara.  Acceleration for both Boltzmann machine learning and mean field theory learning , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[5]  Andreas S. Weigend,et al.  John A. Hertz, Anders S. Krogh, Richard G. Palmer, Introduction to the Theory of Neural Computation , 1993, Artif. Intell..

[6]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[7]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[8]  C. Gielen,et al.  Neural computation and self-organizing maps, an introduction , 1993 .

[9]  M. Hagiwara.  Self-organizing feature map with a momentum term , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[10]  Pierre Demartines,et al.  Kohonen Self-Organizing Maps: Is the Normalization Necessary? , 1992, Complex Syst..

[11]  Michael K. Weir,et al.  A method for self-determination of adaptive learning rates in back propagation , 1991, Neural Networks.

[12]  K. Schulten,et al.  Kohonen's self-organizing maps: exploring their computational capabilities , 1988, IEEE 1988 International Conference on Neural Networks.

[13]  Masafumi Hagiwara,et al.  Theoretical derivation of momentum term in back-propagation , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.