Landscape of a Likelihood Surface for a Gaussian Mixture and its use for the EM Algorithm

The EM algorithm is an efficient algorithm to obtain the ML estimate for incomplete data, but has the local optimality problem. The deterministic annealing EM (DAEM) algorithm was once proposed to solve this problem, which begins a search from the primitive initial point. Then the multi-thread EM (m-EM) algorithm was proposed, which begins the multiple-token EM search from the primitive initial point, resulting in excellent solutions in compensation for a rather heavy computing cost. These previous work indicate the potential of using the primitive initial point as the starting point of the EM algorithm. The paper investigates experimentally the characteristics of a landscape of a likelihood surface around the primitive initial point for a multivariate Gaussian mixture, and based on the observation proposes sensible ways of running the EM algorithm.

[1]  Naonori Ueda,et al.  Deterministic annealing EM algorithm , 1998, Neural Networks.

[2]  Threshold-based multi-thread EM algorithm , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[3]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[4]  Ryohei Nakano,et al.  Threshold-based dynamic annealing for multi-thread DAEM and its extreme , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[5]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[6]  Christophe Biernacki,et al.  Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models , 2003, Comput. Stat. Data Anal..

[7]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.