Unsupervised selection and estimation of finite mixture models

We describe a method for fitting mixture models to multivariate data which performs component selection and does not require external initialization. The novelty of our approach includes: an MML-like (minimum message length) model selection criterion; inclusion of the criterion into the expectation-maximization (EM) algorithm (increasing its ability to escape from local maxima); an initialization strategy supported on the interpretation of EM as a self-annealing algorithm.

[1]  J RobertsStephen,et al.  Bayesian Approaches to Gaussian Mixture Modeling , 1998 .

[2]  P. Tavan,et al.  Deterministic annealing for density estimation by multivariate normal mixtures , 1997 .

[3]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[4]  Alfred O. Hero,et al.  Kullback proximal algorithims for maximum-likelihood estimation , 2000, IEEE Trans. Inf. Theory.

[5]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  David J. Hand,et al.  Mixture Models: Inference and Applications to Clustering , 1989 .

[7]  Anand Rangarajan.  Self Annealing: Unifying Deterministic Annealing and Relaxation Labelling , 1997, EMMCVPR.

[8]  William D. Penny,et al.  Bayesian Approaches to Gaussian Mixture Modeling , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  K. Rose.  Deterministic annealing for clustering, compression, classification, regression, and related optimization problems , 1998, Proc. IEEE.

[10]  C. S. Wallace,et al.  Unsupervised Learning Using MML , 1996, ICML.

[11]  Geoffrey E. Hinton,et al.  SMEM Algorithm for Mixture Models , 1998, Neural Computation.

[12]  R. Cox,et al.  Journal of the Royal Statistical Society B , 1972 .

[13]  Naonori Ueda,et al.  Deterministic annealing EM algorithm , 1998, Neural Networks.

[14]  Matthew Brand,et al.  Structure Learning in Conditional Probability Models via an Entropic Prior and Parameter Extinction , 1999, Neural Computation.

[15]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[16]  HofmannThomas,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997 .

[17]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[18]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[19]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[20]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[21]  José M. N. Leitão,et al.  On Fitting Mixture Models , 1999, EMMCVPR.