Fuzzy c-means clustering with regularization by K-L information

The Gaussian mixture model or Gaussian mixture density decomposition(GMDD) use the likelihood function as a measure of fit. We show that just the same algorithm as the GMDD can be derived from a modified objective function of fuzzy c-means (FCM) clustering with the regularizer by K-L information, only when the parameter /spl lambda/ equals 2. Although the fixed-point iteration scheme of FCM is similar to that of the GMDD, the FCM has more flexible structure since the algorithm is based on the objective function method. In a slightly different manner such as installing a deterministic annealing or an addition of Gustafson and Kessel's (1979) constraint, the proposed algorithm is likely to provide more valid clustering results.

[1]  I Burhan Turksen.  SOFT in the New Millennium , 2000 .

[2]  Hidetomo Ichihashi,et al.  Simultaneous Approach to Fuzzy Clustering, Principal Component and Multiple Regression Analysis , 2000 .

[3]  V. J. Rayward-Smith,et al.  Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition , 1999 .

[4]  J. C. Peters,et al.  Fuzzy cluster analysis--a new method to predict future cardiac events in patients with positive stress tests. , 1998, Japanese circulation journal.

[5]  Naonori Ueda,et al.  Deterministic annealing EM algorithm , 1998, Neural Networks.

[6]  Sadaaki Miyamoto,et al.  Fuzzy Classification Functions in the Methods of Fuzzy c-Means and Regularization by Entropy , 1998 .

[7]  Sadaaki Miyamoto,et al.  Fuzzy c-means as a regularization and maximum entropy approach , 1997 .

[8]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[9]  Xinhua Zhuang,et al.  Gaussian mixture density modeling, decomposition, and applications , 1996, IEEE Trans. Image Process..

[10]  Roy L. Streit,et al.  Maximum likelihood training of probabilistic neural networks , 1994, IEEE Trans. Neural Networks.

[11]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[12]  Rajesh N. Davé,et al.  Characterization and detection of noise in clustering , 1991, Pattern Recognit. Lett..

[13]  Geoffrey C. Fox,et al.  A deterministic annealing approach to clustering , 1990, Pattern Recognit. Lett..

[14]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[17]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[18]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[19]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[20]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[21]  James C. Bezdek,et al.  Fuzzy mathematics in pattern classification , 1973 .