Self-Organization as an Iterative Kernel Smoothing Process

Kohonen's self-organizing map, when described in a batch processing mode, can be interpreted as a statistical kernel smoothing problem. The batch SOM algorithm consists of two steps. First, the training data are partitioned according to the Voronoi regions of the map unit locations. Second, the units are updated by taking weighted centroids of the data falling into the Voronoi regions, with the weighing function given by the neighborhood. Then, the neighborhood width is decreased and steps 1, 2 are repeated. The second step can be interpreted as a statistical kernel smoothing problem where the neighborhood function corresponds to the kernel and neighborhood width corresponds to kernel span. To determine the new unit locations, kernel smoothing is applied to the centroids of the Voronoi regions in the topological space. This interpretation leads to some new insights concerning the role of the neighborhood and dimensionality reduction. It also strengthens the algorithm's connection with the Principal Curve algorithm. A generalized self-organizing algorithm is proposed, where the kernel smoothing step is replaced with an arbitrary nonparametric regression method.

[1]  G.R. De Haan,et al.  Links between self-organizing feature maps and weighted vector quantization , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[2]  Vladimir Cherkassky,et al.  Comparison of adaptive methods for function estimation from samples , 1996, IEEE Trans. Neural Networks.

[3]  Teuvo Kohonen,et al.  Things you haven't heard about the self-organizing map , 1993, IEEE International Conference on Neural Networks.

[4]  E. Nadaraya.  On Estimating Regression , 1964 .

[5]  Alexander Grunewald.  Neighborhoods and trajectories in Kohonen maps , 1992, Defense, Security, and Sensing.

[6]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[7]  Helge J. Ritter,et al.  Neural computation and self-organizing maps - an introduction , 1992, Computation and neural systems series.

[8]  T. Hastie.  Principal Curves and Surfaces , 1984 .

[9]  L. Rutkowski.  Nonparametric identification of quasi-stationary systems , 1985 .

[10]  Jerome H. Friedman,et al.  Smoothing of Scatterplots , 1982 .

[11]  W. Härdle.  Applied Nonparametric Regression: Introduction , 1990 .

[12]  Simon Haykin,et al.  Neural networks expand SP's horizons , 1996, IEEE Signal Process. Mag..

[13]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[14]  Vladimir Cherkassky,et al.  Adaptive knot Placement for Nonparametric Regression , 1993, NIPS.

[15]  I. Ahmad,et al.  NONPARAMETRIC SEQUENTIAL ESTIMATION OF A MULTIPLE REGRESSION FUNCTION , 1976 .

[16]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[17]  Stephen P. Luttrell,et al.  Derivation of a class of training algorithms , 1990, IEEE Trans. Neural Networks.

[18]  Akio Utsugi.  Hyperparameter Selection for Self-Organizing Maps , 1997, Neural Computation.

[19]  S. P. Luttrell.  Self-organising multilayer topographic mappings , 1988, IEEE 1988 International Conference on Neural Networks.

[20]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.