Estimation of Structured Gaussian Mixtures: The Inverse EM Algorithm

This contribution is devoted to the estimation of the parameters of multivariate Gaussian mixture where the covariance matrices are constrained to have a linear structure such as Toeplitz, Hankel, or circular constraints. We propose a simple modification of the expectation-maximization (EM) algorithm to take into account the structure constraints. The basic modification consists of virtually updating the observed covariance matrices in a first stage. Then, in a second stage, the estimated covariances undergo the reversed updating. The proposed algorithm is called the inverse EM algorithm. The increasing property of the likelihood through the algorithm iterations is proved. The strict increasing for nonstationary points is proved as well. Numerical results are shown to corroborate the effectiveness of the proposed algorithm for the joint unsupervised classification and spectral estimation of stationary autoregressive time series.

[1]  R. A. Boyles.  On the Convergence of the EM Algorithm , 1983 .

[2]  Volker Tresp,et al.  Averaging, maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates , 1998, IEEE Trans. Neural Networks.

[3]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .

[4]  R. Hathaway.  A constrained EM algorithm for univariate normal mixtures , 1986 .

[5]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[6]  Hichem Snoussi,et al.  Bayesian Unsupervised Learning for Source Separation with Mixture of Gaussians Prior , 2004, J. VLSI Signal Process..

[7]  P. Nurmi.  Mixture Models , 2008 .

[8]  Josiane Zerubia,et al.  Estimation of Markov random field prior parameters using Markov chain Monte Carlo maximum likelihood , 1999, IEEE Trans. Image Process..

[9]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[10]  D. Luenberger,et al.  Estimation of structured covariance matrices , 1982, Proceedings of the IEEE.

[11]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[12]  Hichem Snoussi,et al.  MCMC joint separation and segmentation of hidden Markov fields , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[13]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[14]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[15]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .

[16]  Hichem Snoussi,et al.  Penalized maximum likelihood for multivariate Gaussian mixture , 2002 .

[17]  Timothy J. Schulz.  Penalized maximum-likelihood estimation of covariance matrices with linear structure , 1997, IEEE Trans. Signal Process..