Maximum Likelihood Estimates for Gaussian Mixtures Are Transcendental

Gaussian mixture models are central to classical statistics, widely used in the information sciences, and have a rich mathematical structure. We examine their maximum likelihood estimates through the lens of algebraic statistics. The MLE is not an algebraic function of the data, so there is no notion of ML degree for these models. The critical points of the likelihood function are transcendental, and there is no bound on their number, even for mixtures of two univariate Gaussians.

[1]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[2]  A. O. Gelʹfond.  Transcendental and Algebraic Numbers , 1960 .

[3]  H. Teicher.  Identifiability of Finite Mixtures , 1963 .

[4]  A. Baker.  Transcendental Number Theory , 1975 .

[5]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[6]  James A. Reeds,et al.  Asymptotic Number of Roots of Cauchy Location Likelihood Equations , 1985 .

[7]  Adrian E. Raftery,et al.  Enhanced Model-Based Clustering, Density Estimation, and Discriminant Analysis Software: MCLUST , 2003, J. Classif..

[8]  Keisuke Yamazaki,et al.  Kullback Information of Normal Mixture is not an Analytic Function , 2004 .

[9]  Deok-Soo Kim,et al.  Shortest Paths for Disc Obstacles , 2004, ICCSA.

[10]  Donald St. P. Richards,et al.  Counting and locating the solutions of polynomial systems of maximum likelihood equations, I , 2006, J. Symb. Comput..

[11]  Chee Yap,et al.  Decidability of collision between a helical motion and an algebraic motion , 2006 .

[12]  Nathan Srebro.  Are There Local Maxima in the Infinite-Sample Likelihood of Gaussian Mixture Estimation? , 2007, COLT.

[13]  Bernd Sturmfels,et al.  Multivariate Gaussians, semidefinite matrix completion, and convex algebraic geometry , 2009, 0906.3529.

[14]  渡邊 澄夫.  Algebraic geometry and statistical learning theory , 2009 .

[15]  Ankur Moitra,et al.  Settling the Polynomial Learnability of Mixtures of Gaussians , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[16]  Mikhail Belkin,et al.  Polynomial Learning of Distribution Families , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[17]  Elizabeth Gross,et al.  Maximum likelihood degree of variance component models , 2011, 1111.3308.

[18]  Qingqing Huang,et al.  Learning Mixtures of Gaussians in High Dimensions , 2015, STOC.

[19]  Bernd Sturmfels,et al.  Moment Varieties of Gaussian Mixtures , 2015, 1510.04654.