A robust deterministic annealing algorithm for data clustering

In this paper, a novel robust deterministic annealing (RDA) algorithm is developed for data clustering. This method takes advantage of conventional noise clustering (NC) and deterministic annealing (DA) algorithms in terms of the independence of data initialization, the ability to avoid poor local optima, the better performance for unbalanced data, and the robustness against noise and outliers. In addition, a cluster validity criterion, i.e., Vapnik-Chervonenkis (VC)-bound induced index, which is estimated based on the structural risk minimization (SRM) principle, is specifically extended for RDA to determine the optimal cluster number for a given data set. The superiority of the proposed RDA clustering algorithm is supported by experimental results.

[1]  K. Rose.  Deterministic annealing for clustering, compression, classification, regression, and related optimization problems , 1998, Proc. IEEE.

[2]  Stephen J. Roberts,et al.  Minimum-Entropy Data Partitioning Using Reversible Jump Markov Chain Monte Carlo , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Jiang-She Zhang,et al.  Robust clustering by pruning outliers , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[4]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[5]  Rajesh N. Davé,et al.  Robust clustering methods: a unified view , 1997, IEEE Trans. Fuzzy Syst..

[6]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  Miin-Shen Yang,et al.  A similarity-based robust clustering method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[10]  Gerardo Beni,et al.  A Validity Measure for Fuzzy Clustering , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  James C. Bezdek,et al.  On cluster validity for the fuzzy c-means model , 1995, IEEE Trans. Fuzzy Syst..

[12]  Kenneth Rose,et al.  Hierarchical, Unsupervised Learning with Growing via Phase Transitions , 1996, Neural Computation.

[13]  Rajesh N. Dave,et al.  Robust shape detection using fuzzy clustering: practical applications , 1994, CVPR 1994.

[14]  Miin-Shen Yang,et al.  Alternative c-means clustering algorithms , 2002, Pattern Recognit..

[15]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[16]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[17]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[18]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[19]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[21]  Geoffrey C. Fox,et al.  Constrained Clustering as an Optimization Method , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[23]  Hichem Frigui,et al.  The fuzzy c spherical shells algorithm: A new approach , 1992, IEEE Trans. Neural Networks.

[24]  H. L. Le Roy,et al.  Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability; Vol. IV , 1969 .

[25]  R. Davé.  FUZZY SHELL-CLUSTERING AND APPLICATIONS TO CIRCLE DETECTION IN DIGITAL IMAGES , 1990 .

[26]  Qing Song.  A Robust Information Clustering Algorithm , 2005, Neural Computation.

[27]  Rajesh N. Davé,et al.  Characterization and detection of noise in clustering , 1991, Pattern Recognit. Lett..

[28]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[29]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[30]  Klaus Obermayer,et al.  Self-organizing maps: Generalizations and new optimization techniques , 1998, Neurocomputing.

[31]  Rose,et al.  Statistical mechanics and phase transitions in clustering. , 1990, Physical review letters.

[32]  Rajesh N. Davé,et al.  Robust fuzzy clustering of relational data , 2002, IEEE Trans. Fuzzy Syst..

[33]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .