A unified view of probabilistic PCA and regularized linear fuzzy clustering

FCM-type fuzzy clustering approaches are closely related to Gaussian mixture models (GMMs) and the objective function of fuzzy c-means with regularization by K-L information (KFCM) is optimized by an EM-like algorithm. In this paper, we propose to apply probabilistic PCA mixture models to linear clustering following the discussion on the relationship between local PCA and linear fuzzy clustering. Although the proposed method is kind of the constrained model of KFCM, the algorithm includes the fuzzy c-varieties (FCV) algorithm as a special case, and the algorithm can be regarded as a modified FCV algorithm with regularization by K-L information.

[1]  Hidetomo Ichihashi,et al.  Robust Clustering in Fuzzy C-Means with Regularization by Cross Entropy , 2002, FSKD.

[2]  J. C. Peters,et al.  Fuzzy Cluster Analysis : A New Method to Predict Future Cardiac Events in Patients With Positive Stress Tests , 1998 .

[3]  W. Peizhuang.  Pattern Recognition with Fuzzy Objective Function Algorithms (James C. Bezdek) , 1983 .

[4]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[5]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[6]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[7]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Kevin Baker,et al.  Classification of radar returns from the ionosphere using neural networks , 1989 .

[9]  Sadaaki Miyamoto,et al.  Fuzzy c-means as a regularization and maximum entropy approach , 1997 .

[10]  Perry D. Moerland.  A comparison of mixture models for density estimation , 1999 .

[11]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[12]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[13]  Keinosuke Fukunaga,et al.  An Algorithm for Finding Intrinsic Dimensionality of Data , 1971, IEEE Transactions on Computers.

[14]  Hidetomo Ichihashi,et al.  Fuzzy c-means clustering with regularization by K-L information , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[15]  Kenneth G. Manton,et al.  Fuzzy Cluster Analysis , 2005 .

[16]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[17]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[18]  Sam T. Roweis,et al.  EM Algorithms for PCA and SPCA , 1997, NIPS.

[19]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[20]  J. Bezdek,et al.  Detection and Characterization of Cluster Substructure II. Fuzzy c-Varieties and Convex Combinations Thereof , 1981 .