A survey of fuzzy clustering algorithms for pattern recognition. I

Clustering algorithms aim at modeling fuzzy (i.e., ambiguous) unlabeled patterns efficiently. Our goal is to propose a theoretical framework where the expressive power of clustering systems can be compared on the basis of a meaningful set of common functional features. Part I of this paper reviews the following issues related to clustering approaches found in the literature: relative (probabilistic) and absolute (possibilistic) fuzzy membership functions and their relationships to the Bayes rule, batch and on-line learning, prototype editing schemes, growing and pruning networks, modular network architectures, topologically perfect mapping, ecological nets and neuro-fuzziness. From this discussion an equivalence between the concepts of fuzzy clustering and soft competitive learning in clustering algorithms is proposed as a unifying framework in the comparison of clustering systems. Moreover, a set of functional attributes is selected for use as dictionary entries in the comparison of clustering algorithms, which is the subject of part II of this paper.

[1]  F. Parmiggiani,et al.  Fuzzy combination of Kohonen's and ART neural network models to detect statistical regularities in a random sequence of multi-valued input patterns , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[2]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[3]  Giuseppe Satalino,et al.  FEATURE EXTRACTION AND PATTERN CLASSIFICATION OF REMOTE SENSING DATA BY A MODULAR NEURAL SYSTEM , 1996 .

[4]  C. Gielen,et al.  Neural computation and self-organizing maps, an introduction , 1993 .

[5]  Sunanda Mitra,et al.  Integrated adaptive fuzzy clustering (IAFC) algorithm , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[6]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[7]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[8]  Rajesh N. Davé,et al.  Robust clustering methods: a unified view , 1997, IEEE Trans. Fuzzy Syst..

[9]  James C. Bezdek,et al.  Multiple-prototype classifier design , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[10]  S. P. Luttrell,et al.  A Bayesian Analysis of Self-Organizing Maps , 1994, Neural Computation.

[11]  Jenlong Moh,et al.  A new art-based neural architecture for pattern classification and image enhancement without prior knowledge , 1992, Pattern Recognit..

[12]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[13]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[14]  C. Kim,et al.  A structure of fuzzy decision-making system for power system protection , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[15]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[16]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[17]  Marvin Minsky,et al.  Perceptrons: expanded edition , 1988 .

[18]  Andrea Baraldi,et al.  Fuzzy clustering: critical analysis of the contextual mechanisms employed by three neural network models , 1996, Defense + Commercial Sensing.

[19]  Christopher M. Bishop,et al.  GTM: A Principled Alternative to the Self-Organizing Map , 1996, NIPS.

[20]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[21]  S. Grossberg,et al.  ART 2: self-organization of stable category recognition codes for analog input patterns. , 1987, Applied optics.

[22]  Sankar K. Pal,et al.  Self-organizing neural network as a fuzzy classifier , 1994, IEEE Trans. Syst. Man Cybern..

[23]  Lorenzo Bruzzone,et al.  A technique for the selection of kernel-function parameters in RBF neural networks for classification of remote-sensing images , 1999, IEEE Trans. Geosci. Remote. Sens..

[24]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[25]  James R. Williamson,et al.  A Constructive, Incremental-Learning Network for Mixture Modeling and Classification , 1997, Neural Computation.

[26]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[27]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[28]  Paolo Frasconi,et al.  Learning without local minima in radial basis function networks , 1995, IEEE Trans. Neural Networks.

[29]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[30]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks - Part 2: Clustering , 1993, IEEE Trans. Fuzzy Syst..

[31]  Ethem Alpaydin,et al.  Soft vector quantization and the EM algorithm , 1998, Neural Networks.

[32]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[33]  L. Eon Bottou.  Online Learning and Stochastic Approximations , 1998 .

[34]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[35]  Yi Zheng,et al.  The effect of concave and convex weight adjustments on self-organizing maps , 1996, IEEE Trans. Neural Networks.

[36]  Marco Gori,et al.  Optimal learning in artificial neural networks: A review of theoretical results , 1996, Neurocomputing.

[37]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[38]  James C. Bezdek,et al.  An integrated approach to fuzzy learning vector quantization and fuzzy c-means clustering , 1997, IEEE Trans. Fuzzy Syst..

[39]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[41]  James C. Bezdek,et al.  Generalized clustering networks and Kohonen's self-organizing scheme , 1993, IEEE Trans. Neural Networks.

[42]  GemanStuart,et al.  Neural networks and the bias/variance dilemma , 1992 .

[43]  Palma Blonda,et al.  Model transitions in descending FLVQ , 1998, IEEE Trans. Neural Networks.

[44]  Roberto Serra,et al.  Complex Systems and Cognitive Processes , 1990, Springer Berlin Heidelberg.

[45]  Ethem Alpaydin,et al.  Simplified ART: A new class of ART algorithms , 1998 .

[46]  L. Schenato,et al.  Soft-to-Hard Model Transition in Clustering: A Review , 1999 .

[47]  Sheng-Fuu Lin,et al.  Adaptive hamming net: A fast-learning ART 1 model without searching , 1995, Neural Networks.

[48]  Léon Bottou,et al.  On-line learning and stochastic approximations , 1999 .

[49]  Mauro Barni,et al.  Comments on "A possibilistic approach to clustering" , 1996, IEEE Trans. Fuzzy Syst..

[50]  Yehoshua Y. Zeevi,et al.  The Generalized Gabor Scheme of Image Representation in Biological and Machine Vision , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[52]  James C. Bezdek,et al.  Repairs to GLVQ: a new family of competitive learning schemes , 1996, IEEE Trans. Neural Networks.

[53]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[54]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[55]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[56]  Stefano Ferrari,et al.  Image reconstruction using improved "neural-gas" , 1996 .

[57]  Joachim M. Buhmann,et al.  Competitive learning algorithms for robust vector quantization , 1998, IEEE Trans. Signal Process..

[58]  James C. Bezdek,et al.  Fuzzy Kohonen clustering networks , 1994, Pattern Recognit..

[59]  Timothy Masters.  Signal and Image Processing with Neural Networks: A C++ Sourcebook , 1994 .

[60]  Geoffrey C. Fox,et al.  A deterministic annealing approach to clustering , 1990, Pattern Recognit. Lett..

[61]  M. Steriade.  The Mind-Brain Continuum edited by Rodolfo Llinás and Patricia S. Churchland, MIT Press, 1996. $50.00 (xi + 315 pages) ISBN 0 262 12198 0 , 1997, Trends in Neurosciences.

[62]  Stephen Grossberg,et al.  Art 2: Self-Organization Of Stable Category Recognition Codes For Analog Input Patterns , 1988, Other Conferences.

[63]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[64]  Michael Georgiopoulos,et al.  Fuzzy ART properties , 1995, Neural Networks.

[65]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[66]  Stephen P. Luttrell,et al.  Derivation of a class of training algorithms , 1990, IEEE Trans. Neural Networks.

[67]  James C. Bezdek,et al.  Two soft relatives of learning vector quantization , 1995, Neural Networks.

[68]  Bart L. M. Happel,et al.  Design and evolution of modular neural network architectures , 1994, Neural Networks.

[69]  Sandro Ridella,et al.  On the importance of sorting in "neural gas" training of vector quantizers , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[70]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[71]  Nicolaos B. Karayiannis,et al.  Fuzzy algorithms for learning vector quantization , 1996, IEEE Trans. Neural Networks.

[72]  Andrea Baraldi,et al.  Novel neural network model combining radial basis function, competitive Hebbian learning rule, and fuzzy simplified adaptive resonance theory , 1997, Optics & Photonics.

[73]  Masao Kasahara,et al.  A construction of vector quantizers for noisy channels , 1984 .