Clustering and Neural Network Approaches

This paper describes how clustering problems can be resolved by neural network (NN) approaches such as Hopfield nets, multi-layer perceptrons, and Kohonen’s ’self-organizing maps’ (SOMs). We emphasize the close relationship between the NN approach and classical clustering methods. In particular, we show how SOMs are derived by stochastic approximation from a new continuous version (K-criterion) of a finite-sample clustering criterion proposed by Anouar et al. (1997). In this framework we determine the asymptotic behaviour of Kohonen’s method, design a new finite-sample version of the SOM approach of the k-means type, and propose various generalizations along the lines of classical ’regression clustering’, ’principal component clustering’, and ’maximum-likelihood clustering’.

[1]  Samuel Kaski,et al.  The Adaptive-Subspace Self-Organizing Map (ASSOM) , 1997 .

[2]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[3]  D. Pollard.  Strong Consistency of $K$-Means Clustering , 1981 .

[4]  Roger N. Shepard,et al.  Additive clustering: Representation of similarities as combinations of discrete overlapping properties. , 1979 .

[5]  H. Bock.  Probabilistic models in cluster analysis , 1996 .

[6]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[7]  M. L. Emrich,et al.  Workshop on neural networks , 1990 .

[8]  G. Govaert,et al.  Constrained clustering and Kohonen Self-Organizing Maps , 1996 .

[9]  G. De Soete,et al.  Clustering and Classification , 2019, Data-Driven Science and Engineering.

[10]  D. Hand.  Cluster dissection and analysis: Helmuth SPATH Wiley, Chichester, 1985, 226 pages, £25.00 , 1986 .

[11]  Hans-Hermann Bock,et al.  Advances in data science and classification , 1998 .

[12]  Hans-Hermann Bock,et al.  Clustering and Neural Networks , 1998 .

[13]  M. Cottrell,et al.  Etude d'un processus d'auto-organisation , 1987 .

[14]  G. A. Butler,et al.  A vector field approach to cluster analysis , 1969, Pattern Recognit..

[15]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Yoshiharu Sato,et al.  Additive Clustering Model and Its Generalization , 1998 .

[17]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[18]  Hans-Hermann Bock,et al.  Data Science, Classification and Related Methods , 1998 .

[19]  Fionn Murtagh,et al.  NEURAL NETWORKS FOR CLUSTERING , 1996 .

[20]  Hans-Hermann Bock,et al.  Simultaneous Visualization and Clustering Methods as an Alternative to Kohonen Maps , 1997 .

[21]  Hans-Hermann Bock,et al.  PROBABILITY MODELS AND HYPOTHESES TESTING IN PARTITIONING CLUSTER ANALYSIS , 1996 .

[22]  Helge Ritter.  NEURAL NETWORKS FOR RAPID LEARNING IN COMPUTER VISION AND ROBOTICS , 1997 .

[23]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[24]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[25]  F. Murtagh,et al.  The Kohonen self-organizing map method: An assessment , 1995 .

[26]  James S. Coleman,et al.  Clustering in N dimensions by use of a system of forces , 1971 .

[27]  Fionn Murtagh,et al.  Clustering Based on Neural Network Processing , 1988 .

[28]  M. Sato,et al.  Neural clustering-implementation of clustering model using neural networks , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[29]  Fionn Murtagh,et al.  Cluster Dissection and Analysis: Theory, Fortran Programs, Examples. , 1986 .

[30]  G. Pagès,et al.  Self-organization and a.s. convergence of the one-dimensional Kohonen algorithm with non-uniformly distributed stimuli , 1993 .

[31]  Rudolf Kruse,et al.  Learning, Networks and Statistics , 1997, International Centre for Mechanical Sciences.

[32]  D. Pollard.  A Central Limit Theorem for $k$-Means Clustering , 1982 .

[33]  H. Bock.  On some significance tests in cluster analysis , 1985 .

[34]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[35]  F. H. C. Marriott,et al.  Analyse de Donnees et Informatique , 1982 .

[36]  Gilles Pagès,et al.  About the Kohonen algorithm: strong or weak self-organization? , 1996, ESANN.