论文信息 - Deterministic Annealing Variant of the EM Algorithm

Deterministic Annealing Variant of the EM Algorithm

We present a deterministic annealing variant of the EM algorithm for maximum likelihood parameter estimation problems. In our approach, the EM process is reformulated as the problem of minimizing the thermodynamic free energy by using the principle of maximum entropy and statistical mechanics analogy. Unlike simulated annealing approaches, this minimization is deterministically performed. Moreover, the derived algorithm, unlike the conventional EM algorithm, can obtain better estimates free of the initial parameter values.

Naonori Ueda | Ryohei Nakano | N. Ueda | R. Nakano

[1] Yiu-Fai Wong,et al. Clustering Data by Melting , 1993, Neural Computation.

[2] Donald Geman,et al. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3] Radford M. Neal. A new view of the EM algorithm that justifies incremental and other variants , 1993 .

[4] R. Hathaway. Another interpretation of the EM algorithm for mixture distributions , 1986 .

[5] Geoffrey C. Fox,et al. Vector quantization by deterministic annealing , 1992, IEEE Trans. Inf. Theory.

[6] Alan L. Yuille,et al. Statistical Physics, Mixtures of Distributions, and the EM Algorithm , 1994, Neural Computation.

[7] N. Ueda,et al. Mixture density estimation via EM algorithm with deterministic annealing , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[8] Joachim M. Buhmann,et al. Complexity Optimized Data Clustering by Competitive Neural Networks , 1993, Neural Computation.

[9] Robert A. Jacobs,et al. Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[10] D. Rubin,et al. Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[11] William J. Byrne,et al. Alternating minimization and Boltzmann machine learning , 1992, IEEE Trans. Neural Networks.