Entropy-type classification maximum likelihood algorithms for mixture models

Mixtures of distributions are popularly used as probability models for analyzing grouped data. Classification maximum likelihood (CML) is an important maximum likelihood approach to clustering with mixture models. Yang et al. extended CML to fuzzy CML. Although fuzzy CML presents better results than CML, it is always affected by the fuzziness index parameter. In this paper, we consider fuzzy CML with an entropy-regularization term to create an entropy-type CML algorithm. The proposed entropy-type CML is a parameter-free algorithm for mixture models. Some numerical and real-data comparisons show that the proposed method provides better results than some existing methods.

[1]  Doheon Lee,et al.  Fuzzy clustering of categorical data using fuzzy centroids , 2004, Pattern Recognit. Lett..

[2]  Miin-Shen Yang.  On a class of fuzzy classification maximum likelihood procedures , 1993 .

[3]  Miin-Shen Yang,et al.  A fuzzy k-partitions model for categorical data and its comparison to the GoM model , 2008, Fuzzy Sets Syst..

[4]  B. Everitt,et al.  Finite Mixture Distributions , 1981 .

[5]  Joshua Zhexue Huang,et al.  Extensions to the k-Means Algorithm for Clustering Large Data Sets with Categorical Values , 1998, Data Mining and Knowledge Discovery.

[6]  D. N. Geary.  Mixture Models: Inference and Applications to Clustering , 1989 .

[7]  Michael J. Symons,et al.  Clustering criteria and multivariate normal mixtures , 1981 .

[8]  Miin-Shen Yang.  A survey of fuzzy clustering , 1993 .

[9]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[10]  Jian Yu,et al.  A Generalized Fuzzy Clustering Regularization Model With Optimality Tests and Model Complexity Analysis , 2007, IEEE Transactions on Fuzzy Systems.

[11]  Peter Bryant,et al.  Asymptotic behaviour of classification maximum likelihood estimates , 1978 .

[12]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[13]  A. Scott,et al.  Clustering methods based on likelihood ratio criteria. , 1971 .

[14]  V. J. Rayward-Smith,et al.  Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition , 1999 .

[15]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[16]  Michael K. Ng,et al.  A fuzzy k-modes algorithm for clustering categorical data , 1999, IEEE Trans. Fuzzy Syst..

[17]  E. Forgy,et al.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[18]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[19]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[20]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[21]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .