Dynamic mixtures of finite mixtures and telescoping sampling

Within a Bayesian framework, a comprehensive investigation of the model class of mixtures of finite mixtures (MFMs) where a prior on the number of components is specified is performed. This model class has applications in model-based clustering as well as for semi-parametric density approximation, but requires suitable prior specifications and inference methods to exploit its full potential. We contribute to the Bayesian analysis of MFMs by (1) considering static and dynamic MFMs where the Dirichlet parameter of the component weights either is fixed or depends on the number of components, (2) proposing a flexible prior distribution class for the number of components $K$, (3) characterizing the implicit prior on the number of clusters $K_+$ as well as partitions by deriving computationally feasible formulas, (4) linking MFMs to Bayesian non-parametric mixtures, and (5) finally proposing a novel sampling scheme for MFMs called the telescoping sampler which allows Bayesian inference for mixtures with arbitrary component distributions. The telescoping sampler explicitly samples the number of components, but otherwise requires only the usual MCMC steps for estimating a finite mixture model. The ease of its application using different component distributions is demonstrated on real data sets.

[1]  T. Sweeting,et al.  Selecting the precision parameter prior in Dirichlet process mixture models , 2012 .

[2]  M. Aitkin.  Likelihood and Bayesian analysis of mixtures , 2001 .

[3]  J. Pitman,et al.  The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator , 1997 .

[4]  A. Gnedin.  A Species Sampling Model with Finitely Many Types , 2009, 0910.1988.

[5]  Gertraud Malsiner-Walli,et al.  Identifying Mixtures of Mixtures Using Bayesian Estimation , 2015, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[6]  J. Pitman.  Exchangeable and partially exchangeable random partitions , 1995 .

[7]  Luca Scrucca,et al.  mclust 5: Clustering, Classification and Density Estimation Using Gaussian Finite Mixture Models , 2016, R J..

[8]  D. Rubin,et al.  STATISTICAL CHOICES IN INFANT TEMPERAMENT RESEARCH , 1994 .

[9]  J. Pitman,et al.  Exchangeable Gibbs partitions and Stirling triangles , 2004, math/0412494.

[10]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[11]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[12]  P. Green,et al.  Modelling Heterogeneity With and Without the Dirichlet Process , 2001 .

[13]  Walter R. Gilks,et al.  Bayesian model comparison via jump diffusions , 1995 .

[14]  Michael A. West,et al.  Computing Nonparametric Hierarchical Models , 1998 .

[15]  Agostino Nobile,et al.  On the posterior distribution of the number of components in a finite mixture , 2004, math/0503673.

[16]  María José García-Zattera,et al.  A Dirichlet process mixture model for the analysis of correlated binary responses , 2007, Comput. Stat. Data Anal..

[17]  Jeffrey W. Miller,et al.  Mixture Models With a Prior on the Number of Components , 2015, Journal of the American Statistical Association.

[18]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[19]  Gertraud Malsiner-Walli,et al.  From here to infinity: sparse finite versus Dirichlet process mixtures in model-based clustering , 2018, Advances in Data Analysis and Classification.

[20]  P. Green,et al.  Corrigendum: On Bayesian analysis of mixtures with an unknown number of components , 1997 .

[21]  Gertraud Malsiner-Walli,et al.  Model-based clustering based on sparse finite Gaussian mixtures , 2014, Statistics and Computing.

[22]  R. Dorazio.  On selecting a prior for the precision parameter of Dirichlet process mixture models , 2009 .

[23]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[24]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[25]  Matthew T. Harrison,et al.  A simple example of Dirichlet process mixture inconsistency for the number of components , 2013, NIPS.

[26]  Eric R. Ziegel,et al.  Practical Nonparametric and Semiparametric Bayesian Statistics , 1998, Technometrics.

[27]  Agostino Nobile,et al.  Bayesian finite mixtures with an unknown number of components: The allocation sampler , 2007, Stat. Comput..

[28]  Joshua Zhexue Huang,et al.  A Fast Clustering Algorithm to Cluster Very Large Categorical Data Sets in Data Mining , 1997, DMKD.

[29]  Cristiano Villa,et al.  On a loss-based prior for the number of components in mixture models , 2018, Statistics & Probability Letters.

[30]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[31]  Matteo Ruggiero,et al.  Are Gibbs-Type Priors the Most Natural Generalization of the Dirichlet Process? , 2015, IEEE transactions on pattern analysis and machine intelligence.

[32]  K. Roeder.  Density estimation with confidence sets exemplified by superclusters and voids in the galaxies , 1990 .

[33]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .