An Iterative Entropy Regularized Likelihood Learning Algorithm for Cluster Analysis with the Number of Clusters Automatically Detected

As for cluster analysis, the key problem is to determine the number of clusters. This paper presents an entropy regularized likelihood (ERL) learning principle for cluster analysis based on a mixture model to solve this problem. The well-known maximum likelihood estimation is just a special case of ERL learning. Moreover, when the regularization term is the same important as the log-likelihood, the ERL learning actually becomes Bayesian Ying-Yang (BYY) harmony learning. An iterative implementation is then proposed for ERL learning instead of gradient descent, the simulation and color image segmentation experiments have demonstrated that the iterative ERL learning algorithm can automatically detect the number of clusters during the parameter estimation

[1]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[2]  Zhiwu Lu,et al.  A Gradient BYY Harmony Learning Algorithm on Mixture of Experts for Curve Detection , 2005, IDEAL.

[3]  Lei Xu,et al.  BYY harmony learning, structural RPCL, and topological self-organizing on mixture models , 2002, Neural Networks.

[4]  H. Akaike.  A new look at the statistical model identification , 1974 .

[5]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[6]  James C. Bezdek,et al.  Some new indexes of cluster validity , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[7]  N. Boujemaa.  Generalized competitive clustering for image segmentation , 2000, PeachFuzz 2000. 19th International Conference of the North American Fuzzy Information Processing Society - NAFIPS (Cat. No.00TH8500).

[8]  Jinwen Ma,et al.  A gradient BYY harmony learning rule on Gaussian mixture with automated model selection , 2004, Neurocomputing.

[9]  Anil K. Jain,et al.  A self-organizing network for hyperellipsoidal clustering (HEC) , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[10]  Lei Xu,et al.  RBF nets, mixture experts, and Bayesian Ying-Yang learning , 1998, Neurocomputing.

[11]  C. L. Philip Chen,et al.  Cluster number selection for a small set of samples using the Bayesian Ying-Yang model , 2002, IEEE Trans. Neural Networks.