Hill-Climbing, Density-Based Clustering and Equiprobabilistic Topographic Maps

A new approach to density-based clustering and unsupervised classification is introduced for topographic maps. By virtue of our topographic map learning rule, called the kernel-based Maximum Entropy learning Rule (kMER), all neurons have an equal probability to be active (equiprobabilistic map) and, in addition, pilot density estimates are obtained that are compatible with the variable kernel density estimation method. The neurons receive their cluster labels by performing hill-climbing on the density estimates which are located at the neuron weights only. Several methods are suggested and explored for determining the cluster boundaries and the clustering performance is tested for the case where the cluster regions are used for unsupervised classification purposes. Finally, the difference is indicated between (Gaussian) kernel-based density modeling with kMER, and (Gaussian) mixture modeling with maximum likelihood learning.

[1]  Dominique Martinez,et al.  On an Unsupervised Learning Rule for Scalar Quantization following the Maximum Entropy Principle , 1993, Neural Computation.

[2]  Helge Ritter.  Asymptotic level density for a class of vector quantization processes , 1991, IEEE Trans. Neural Networks.

[3]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[4]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[5]  Gilles Pagès.  Voronoï tesselation, space quantization algorithms and numerical integration , 1993, ESANN.

[6]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[7]  Torsten Kuhlen,et al.  Comparative analysis of fuzzy ART and ART-2A network clustering performance , 1998, IEEE Trans. Neural Networks.

[8]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[9]  D. E. Van den Bout,et al.  TInMANN: the integer Markovian artificial neural network , 1989, International 1989 Joint Conference on Neural Networks.

[10]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .

[11]  Stanley C. Ahalt,et al.  Codeword distribution for frequency sensitive competitive learning with one-dimensional input data , 1996, IEEE Trans. Neural Networks.

[12]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[13]  T. Kohonen.  Self-Organized Formation of Correct Feature Maps , 1982 .

[14]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[15]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .

[16]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[17]  Teuvo Kohonen,et al.  Self-organization and associative memory: 3rd edition , 1989 .

[18]  Yeuvo Jphonen,et al.  Self-Organizing Maps , 1995 .

[19]  Y. Cheng,et al.  Clustering with competing self-organizing maps , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[20]  M. Stone,et al.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[21]  Van Hulle MM.  Kernel-Based Equiprobabilistic Topographic Map Formation. , 1998, Neural computation.

[22]  Michel Herbin,et al.  A clustering method based on the estimation of the probability density function and on the skeleton by influence zones. Application to image processing , 1996, Pattern Recognit. Lett..

[23]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[24]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[25]  R. Nakano,et al.  A new learning approach based on equidistortion principle for optimal vector quantizer design , 1993, Neural Networks for Signal Processing III - Proceedings of the 1993 IEEE-SP Workshop.

[26]  Dominik R. Dersch,et al.  Asymptotic level density in topological feature maps , 1995, IEEE Trans. Neural Networks.

[27]  D. Martinez,et al.  The Offset Algorithm: Building and Learning Method for Multilayer Neural Networks , 1992 .

[28]  K. Obermayer,et al.  PHASE TRANSITIONS IN STOCHASTIC SELF-ORGANIZING MAPS , 1997 .

[29]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[30]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[31]  Ralf Der,et al.  Controlling the Magnification Factor of Self-Organizing Feature Maps , 1996, Neural Computation.

[32]  L. N. Kanal,et al.  Handbook of Statistics, Vol. 2. Classification, Pattern Recognition and Reduction of Dimensionality. , 1985 .

[33]  Keinosuke Fukunaga,et al.  A Nonparametric Valley-Seeking Technique for Cluster Analysis , 1971, IEEE Transactions on Computers.

[34]  K. Leuven,et al.  Clustering with kernel-based equiprobabilistic topographic maps , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[35]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[36]  S. Grossberg,et al.  A self-organizing neural network for supervised learning, recognition, and prediction , 1992, IEEE Communications Magazine.

[37]  Sandro Ridella,et al.  Circular backpropagation networks embed vector quantization , 1999, IEEE Trans. Neural Networks.

[38]  Stanley C. Ahalt,et al.  Competitive learning algorithms for vector quantization , 1990, Neural Networks.

[39]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[40]  Marc M. Van Hulle.  Density-based clustering with topographic maps , 1999, IEEE Trans. Neural Networks.