Asymptotic Convergence Properties of EM-Type Algorithms 1

We analyze the asymptotic convergence properties of a general class of EM-type algorithms for estimating an unknown parameter via alternating estimation and maximization. As examples, this class includes ML-EM, penalized ML-EM, Green's OSL-EM, and many other approximate EM algorithms. A theorem is given which provides conditions for monotone convergence with respect to a given norm and speci es an asymptotic rate of convergence for an algorithm in this class. By investigating di erent parameterizations, the condition for monotone convergence can be used to establish norms under which the distance between successive iterates and the limit point of the EM-type algorithm approaches zero monotonically. We apply these results to a modi ed ML-EM algorithm with stochastic complete/incomplete data mapping and establish global monotone convergence for a linear Gaussian observation model. We then establish that in the nal iterations the unpenalized and quadratically penalized ML-EM algorithms for PET image reconstruction converge monotonically relative to two di erent norms on the logarithm of the images.

[1]  Benjamin Naumann,et al.  Computational Methods In Optimization A Unified Approach , 2016 .

[2]  Evgueni A. Haroutunian,et al.  Information Theory and Statistics , 2011, International Encyclopedia of Statistical Science.

[3]  Alfred O. Hero,et al.  Complete-data spaces and generalized EM algorithms , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  Richard M. Leahy,et al.  Statistic-based MAP image-reconstruction from Poisson data using Gibbs priors , 1992, IEEE Trans. Signal Process..

[5]  Ehud Weinstein,et al.  Estimate-maximize algorithms for multichannel time delay and signal estimation , 1991, IEEE Trans. Signal Process..

[6]  P. Green On Use of the EM Algorithm for Penalized Likelihood Estimation , 1990 .

[7]  P. Green Bayesian reconstructions from emission tomography data using a modified EM algorithm. , 1990, IEEE transactions on medical imaging.

[8]  K. Lange Convergence of EM image reconstruction algorithms with Gibbs smoothing. , 1990, IEEE transactions on medical imaging.

[9]  Ehud Weinstein,et al.  Maximum likelihood noise cancellation using the EM algorithm , 1989, IEEE Trans. Acoust. Speech Signal Process..

[10]  T. Hebert,et al.  A generalized EM algorithm for 3-D Bayesian reconstruction from Poisson data using Gibbs priors. , 1989, IEEE transactions on medical imaging.

[11]  Michael I. Miller,et al.  An EM Algorithm For Direction-Of-Arrival Estimation For Narrowband Signals In Noiset , 1988, Optics & Photonics.

[12]  M.I. Miller,et al.  The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process intensities and toeplitz constrained covariances , 1987, Proceedings of the IEEE.

[13]  K. Lange,et al.  EM reconstruction algorithms for emission and transmission tomography. , 1984, Journal of computer assisted tomography.

[14]  F. Graybill,et al.  Matrices with Applications in Statistics. , 1984 .

[15]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[16]  L. Shepp,et al.  Maximum Likelihood Reconstruction for Emission Tomography , 1983, IEEE Transactions on Medical Imaging.

[17]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[18]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.