Improving the EM algorithm for mixtures

One of the estimating equations of the Maximum Likelihood Estimation method, for finite mixtures of the one parameter exponential family, is the first moment equation. This can help considerably in reducing the labor and the cost of calculating the Maximum Likelihood estimates. In this paper it is shown that the EM algorithm can be substantially improved by using this result when applied for mixture models. A short discussion about other methods proposed for the calculation of the Maximum Likelihood estimates are also reported showing that the above findings can help in this direction too.

[1]  A. W. Kemp,et al.  Univariate Discrete Distributions , 1993 .

[2]  D. Karlis,et al.  Minimum Hellinger Distance Estimation for Poisson Mixtures , 1998 .

[3]  V. Hasselblad.  Finite mixtures of distributions from the exponential family , 1969 .

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  Bruce G. Lindsay,et al.  A review of semiparametric mixture models , 1995 .

[6]  Murray Aitkin,et al.  A hybrid EM/Gauss-Newton algorithm for maximum likelihood in mixture distributions , 1996, Stat. Comput..

[7]  A. W. Kepm.  Weighted discrepancies and maximum likelihood estimation for discrete distributions , 1986 .

[8]  B. Lindsay,et al.  Measuring the relative effectiveness of moment estimators as starting values in maximizing likelihoods , 1994 .

[9]  D. Böhning.  A review of reliable maximum likelihood algorithms for semiparametric mixture models , 1995 .

[10]  I. Good.  Conditions for a quadratic form to have a chi-squared distribution , 1969 .

[11]  D. A. Sprott.  Estimating the Parameters of a Convolution by Maximum Likelihood , 1983 .

[12]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[13]  Bruce G. Lindsay,et al.  Properties of the Maximum Likelihood Estimator of a Mixing Distribution , 1981 .

[14]  B. Lindsay.  Mixture models : theory, geometry, and applications , 1995 .

[15]  B. Lindsay,et al.  The distribution of the likelihood ratio for mixtures of densities from the one-parameter exponential family , 1994 .

[16]  N. E. Day.  Estimating the components of a mixture of normal distributions , 1969 .

[17]  Javad Behboodian,et al.  On a mixture of normal distributions , 1970 .