论文信息 - The Expectation-Maximization and Alternating Minimization Algorithms

The Expectation-Maximization and Alternating Minimization Algorithms

The Expectation-Maximization (EM) algorithm is a hill-climbing approach to finding a local maximum of a likelihood function [7, 8]. The EM algorithm alternates between finding a greatest lower bound to the likelihood function (the “E Step”), and then maximizing this bound (the “M Step”). The EM algorithm belongs to a broader class of alternating minimization algorithms [6], which includes the Arimoto-Blahut algorithm for calculating channel capacity and rate distortion functions [1, 3], and Cover’s portfolio algorithm to maximize expected log-investment [4].

Shane M. Haas

[1] Jeff A. Bilmes,et al. A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[2] Zoubin Ghahramani,et al. A Unifying Review of Linear Gaussian Models , 1999, Neural Computation.

[3] Lawrence R. Rabiner,et al. A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[4] Thomas M. Cover,et al. An algorithm for maximizing expected log investment return , 1984, IEEE Trans. Inf. Theory.

[5] Suguru Arimoto,et al. An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[6] R. Shumway,et al. AN APPROACH TO TIME SERIES SMOOTHING AND FORECASTING USING THE EM ALGORITHM , 1982 .

[7] G. McLachlan,et al. The EM algorithm and extensions , 1996 .

[8] Richard E. Blahut,et al. Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[9] T. Minka. Expectation-Maximization as lower bound maximization , 1998 .

[10] Geoffrey E. Hinton,et al. A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[11] R. Okafor. Maximum likelihood estimation from incomplete data , 1987 .

[12] T. Cover. Universal Portfolios , 1996 .