Generalized Mixtures of Finite Mixtures and Telescoping Sampling

Within a Bayesian framework, a comprehensive investigation of the model class of mixtures of finite mixtures (MFMs) where a prior on the number of components is specified is performed. This model class has applications in model-based clustering as well as for semi-parametric density approximation, but requires suitable prior specifications and inference methods to exploit its full potential. We contribute to the Bayesian analysis of MFMs by considering a generalized class of MFMs containing static and dynamic MFMs where the Dirichlet parameter of the component weights either is fixed or depends on the number of components. We emphasize the distinction between the number of components $K$ of a mixture and the number of clusters $K_+$, i.e., the number of filled components given the data. In the MFM model, $K_+$ is a random variable and its prior depends on the prior on the number of components $K$ and the mixture weights. We characterize the prior on the number of clusters $K_+$ for generalized MFMs and derive computationally feasible formulas to calculate this implicit prior. In addition we propose a flexible prior distribution class for the number of components $K$ and link MFMs to Bayesian non-parametric mixtures. For posterior inference of a generalized MFM, we propose the novel telescoping sampler which allows Bayesian inference for mixtures with arbitrary component distributions without the need to resort to RJMCMC methods. The telescoping sampler explicitly samples the number of components, but otherwise requires only the usual MCMC steps for estimating a finite mixture model. The ease of its application using different component distributions is demonstrated on several data sets.