Constrained monotone EM algorithms for finite mixture of multivariate Gaussians

The likelihood function for normal multivariate mixtures may present both local spurious maxima and also singularities and the latter may cause the failure of the optimization algorithms. Theoretical results assure that imposing some constraints on the eigenvalues of the covariance matrices of the multivariate normal components leads to a constrained parameter space with no singularities and at least a smaller number of local maxima of the likelihood function. Conditions assuring that an EM algorithm implementing such constraints maintains the monotonicity property of the usual EM algorithm are provided. Different approaches are presented and their performances are evaluated and compared using numerical experiments.

[1]  C. Theobald.  An inequality with application to multivariate analysis , 1975 .

[2]  R. Rocci,et al.  Monotone Constrained EM Algorithms for Multinormal Mixture Models , 2006 .

[3]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[4]  R. Hathaway.  A Constrained Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions , 1985 .

[5]  Hichem Snoussi,et al.  Penalized maximum likelihood for multivariate Gaussian mixture , 2002 .

[6]  P. Deb.  Finite Mixture Models , 2008 .

[7]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[8]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[9]  S. Ingrassia.  A likelihood-based constrained algorithm for multivariate normal mixture models , 2004 .

[10]  J. Idier,et al.  Penalized Maximum Likelihood Estimator for Normal Mixtures , 2000 .

[11]  Gérard Govaert,et al.  Gaussian parsimonious clustering models , 1995, Pattern Recognit..

[12]  A. Raftery,et al.  Model-based Gaussian and non-Gaussian clustering , 1993 .

[13]  Xiao-Li Meng,et al.  Maximum likelihood estimation via the ECM algorithm: A general framework , 1993 .

[14]  ScienceDirect.  Computational statistics & data analysis , 1983 .

[15]  Maurizio Vichi,et al.  Data Analysis, Classification and the Forward Search , 2006 .

[16]  Patrick J. F. Groenen,et al.  Data Analysis, Classification and the Forward Search , 2006 .

[17]  Christophe Biernacki,et al.  Degeneracy in the Maximum Likelihood Estimation of Univariate Gaussian Mixtures for Grouped Data and Behaviour of the EM Algorithm , 2007 .

[18]  Adrian E. Raftery,et al.  Model-Based Clustering, Discriminant Analysis, and Density Estimation , 2002 .

[19]  C. Biernacki,et al.  Degeneracy in the maximum likelihood estimation of univariate Gaussian mixtures with EM , 2003 .