Parabolic acceleration of the EM algorithm

A new acceleration scheme for optimization procedures is defined through geometric considerations and applied to the EM algorithm. In many cases it is able to circumvent the problem of stagnation. No modification of the original algorithm is required. It is simply used as a software component. Thus the new scheme can be easily implemented to accelerate a fixed point algorithm maximizing some objective function. Some practical examples and simulations are presented to show its ability to accelerate EM-type algorithms converging slowly.

[1]  Xiao-Li Meng,et al.  The EM Algorithm—an Old Folk‐song Sung to a Fast New Tune , 1997 .

[2]  Gerald Farin,et al.  Geometric modeling : algorithms and new trends , 1987 .

[3]  Ch. Roland,et al.  Acceleration schemes with application to the EM algorithm , 2007, Comput. Stat. Data Anal..

[4]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[5]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[6]  Xiao-Li Meng,et al.  Maximum likelihood estimation via the ECM algorithm: A general framework , 1993 .

[7]  J. Wolfe.  PATTERN CLUSTERING BY MULTIVARIATE MIXTURE ANALYSIS. , 1970, Multivariate behavioral research.

[8]  David E. Tyler,et al.  A curious likelihood identity for the multivariate t-distribution , 1994 .

[9]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[10]  W. J. Langford.  Statistical Methods , 1959, Nature.

[11]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[12]  Christophe Biernacki,et al.  Initializing EM using the properties of its trajectories in Gaussian mixtures , 2004, Stat. Comput..

[13]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[14]  D. Rubin,et al.  Parameter expansion to accelerate EM: The PX-EM algorithm , 1998 .

[15]  R. Jennrich,et al.  Conjugate Gradient Acceleration of the EM Algorithm , 1993 .

[16]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[17]  V. Hasselblad.  Finite mixtures of distributions from the exponential family , 1969 .

[18]  R. Jennrich,et al.  Acceleration of the EM Algorithm by using Quasi‐Newton Methods , 1997 .

[19]  G. McLachlan,et al.  The EM Algorithm and Extensions: Second Edition , 2008 .

[20]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[21]  Eric Moulines,et al.  Inference in Hidden Markov Models (Springer Series in Statistics) , 2005 .

[22]  Claude Brezinski,et al.  Extrapolation methods - theory and practice , 1993, Studies in computational mathematics.

[23]  Christophe Biernacki,et al.  Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models , 2003, Comput. Stat. Data Anal..

[24]  Marianthi Markatou,et al.  Weighted Likelihood Equations with Bootstrap Root Search , 1998 .

[25]  Ch. Roland,et al.  Squared Extrapolation Methods (SQUAREM): A New Class of Simple and Efficient Numerical Schemes for Accelerating the Convergence of the EM Algorithm , 2004 .

[26]  D. Rubin,et al.  The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence , 1994 .