Online k-MLE for Mixture Modeling with Exponential Families

This paper address the problem of online learning finite statistical mixtures of exponential families. A short review of the Expectation-Maximization (EM) algorithm and its online extensions is done. From these extensions and the description of the k-Maximum Likelihood Estimator (k-MLE), three online extensions are proposed for this latter. To illustrate them, we consider the case of mixtures of Wishart distributions by giving details and providing some experiments.

[1]  O. Cappé,et al.  On‐line expectation–maximization algorithm for latent data models , 2009 .

[2]  Frank Nielsen,et al.  Statistical exponential families: A digest with flash cards , 2009, ArXiv.

[3]  Frank Nielsen,et al.  Hartigan’s Method for k-MLE: Mixture Modeling with Wishart Distributions and Its Application to Motion Retrieval , 2014 .

[4]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[5]  Qiang Liu,et al.  Distributed Estimation, Information Loss and Exponential Families , 2014, NIPS.

[6]  Allou Samé,et al.  An online classification EM algorithm based on the mixture model , 2007, Stat. Comput..

[7]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[8]  L. Williams,et al.  Contents , 2020, Ophthalmology (Rochester, Minn.).

[9]  Frank Nielsen,et al.  On learning statistical mixtures maximizing the complete likelihood The k-MLE methodology using geometric hard clustering , 2015 .

[10]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[11]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[12]  J. Wishart.  THE GENERALISED PRODUCT MOMENT DISTRIBUTION IN SAMPLES FROM A NORMAL MULTIVARIATE POPULATION , 1928 .

[13]  G. Celeux,et al.  A Classification EM algorithm for clustering and two stochastic versions , 1992 .

[14]  D. Titterington.  Recursive Parameter Estimation Using Incomplete Data , 1984 .

[15]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .