Statistical mechanics of the maximum-likelihood density estimation.

Estimating the density of data generated by Gaussian mixtures, using the maximum-likelihood criterion, is investigated. Solving the statistical mechanics of this problem we evaluate the quality of the estimation as a function of the number of data points, P=\ensuremath{\alpha}N, N being the dimensionality of the points, in the limit of large N. Below a critical value of \ensuremath{\alpha}, the estimated density consists of Gaussian centers that have zero overlap with the structure of the true mixture. We show numerically that estimating the centers by slowly reducing the estimated Gaussian width yields a good agreement with the theory even in the presence of many local minima.