THE TIME ADAPTIVE SELF-ORGANIZING MAP FOR DISTRIBUTION ESTIMATION

The feature map represented by the set of weight vectors of the basic SOM (Self-Organizing Map) provides a good approximation to the input space from which the sample vectors come. But the time- decreasing learning rate and neighborhood function of the basic SOM algorithm reduce its capability to adapt weights for a varied environment. In dealing with non-stationary input distributions and changing environments, we propose a modified SOM algorithm called "Time Adaptive SOM", or TASOM, that automatically adjusts learning rate and neighborhood function of each neuron independently. Each neuron's learning rate is determined by a function of distance between an input vector and its weight vector. The width of the neighborhood function is updated by a function of the distance between the weight vector of the neuron and the weight vectors of neighboring neurons. Only one time parameter initialization is sufficient throughout the lifetime of TASOM to enable it to work with stationary as well as non-stationary input distributions without the need for retraining. The proposed TASOM is tested with five different input distributions and its performance is compared with that of the basic SOM for these cases. The quantization errors of the TASOM in all of these experiments are lower than the errors of the basic SOM. Moreover, the convergence speed of the TASOM outperforms that of the basic SOM. These experiments demonstrate that the TASOM is stable and convergent. The TASOM network is also tested with non-stationary environments when the input distribution completely changes from one distribution to another. The TASOM in these changing environments moves its weights gradually from the old distribution to the clusters of the new distribution. This property is comparable to the memory of human brain, which gradually forgets old memory and memorizes new sensory data. ﻥﺯﻭ ﻱﺎﻫﺭﺍﺩﺮﺑ ﻪﻋﻮﻤﺠﻣ ﻂﺳﻮﺗ ﻪﻛ ﻲﮔﮋﻳﻭ ﺖﺷﺎﮕﻧ SOM

[1]  Tom Heskes,et al.  Neural networks learning in a changing environment , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[2]  Thomas Villmann,et al.  Applications of the growing self-organizing map , 1998, Neurocomputing.

[3]  Amitava Datta,et al.  A modified self-organizing neural net for shape extraction , 1997, Neurocomputing.

[4]  Erkki Oja,et al.  Self - Organizing Maps and Computer Vision , 1992 .

[5]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[6]  Choong Woong Lee,et al.  Fast competitive learning with classified learning rates for vector quantization , 1995, Signal Process. Image Commun..

[7]  Chau-Yun Hsu,et al.  An improved algorithm for Kohonen's self-organizing feature maps , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[8]  R. Safabakhsh,et al.  Pattern classification by the time adaptive self-organizing map , 2000, ICECS 2000. 7th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.00EX445).

[9]  Liang-Gee Chen,et al.  Vector quantization using tree-structured self-organizing feature maps , 1994, IEEE J. Sel. Areas Commun..

[10]  Karin Haese,et al.  Self-organizing feature maps with self-adjusting learning parameters , 1998, IEEE Trans. Neural Networks.

[11]  Teuvo Kohonen,et al.  The 'neural' phonetic typewriter , 1988, Computer.

[12]  J. Gresser,et al.  Reduced risk of Kohonen's feature map non-convergence by an individual size of the neighborhood , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[13]  Michel Verleysen,et al.  Image compression by self-organized Kohonen map , 1998, IEEE Trans. Neural Networks.

[14]  Reza Safabakhsh,et al.  TASOM: the time adaptive self-organizing map , 2000, Proceedings International Conference on Information Technology: Coding and Computing (Cat. No.PR00540).

[15]  Risto Miikkulainen,et al.  Incremental grid growing: encoding high-dimensional structure into a two-dimensional feature map , 1993, IEEE International Conference on Neural Networks.

[16]  Helge J. Ritter,et al.  Neural computation and self-organizing maps - an introduction , 1992, Computation and neural systems series.

[17]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[18]  Reza Safabakhsh,et al.  AUTOMATIC ADJUSTMENT OF LEARNING RATES OF THE SELF-ORGANIZING FEATURE MAP , 2001 .

[19]  Jorma Laaksonen,et al.  Variants of self-organizing maps , 1990, International 1989 Joint Conference on Neural Networks.

[20]  FritzkeBernd.  Growing cell structuresa self-organizing network for unsupervised and supervised learning , 1994 .

[21]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[22]  T. Trautmann,et al.  Comparison of dynamic feature map models for environmental monitoring , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.