Provably Learning Mixtures of Gaussians and More

Given a random sample, how can one accurately estimate the parameters of the probabilistic model that generated the data? This is a fundamental question in statistical inference and, more recently, in machine learning. One of the most widely studied instances of this problem is estimating the parameters of a mixture of Gaussians, since doing so is of fundamental importance in a wide range of subjects, from physics to social science. The classic, and most popular approach, for learning Gaussian mixture models (GMMs) is the EM algorithm [Dempster et al., 1977]. The EM algorithm is a search heuristic over the parameters that finds a local maximum of the likelihood function, and therefore makes no guarantees that it will converge to an estimate that is close to the true parameters. Furthermore, in practice it has been found to converge very slowly.