MCMC for Normalized Random Measure Mixture Models

This paper concerns the use of Markov chain Monte Carlo methods for posterior sampling in Bayesian nonparametric mixture models with normalized random measure priors. Making use of some recent posterior characterizations for the class of normalized random measures, we propose novel Markov chain Monte Carlo methods of both marginal type and conditional type. The proposed marginal samplers are generalizations of Neal's well-regarded Algorithm 8 for Dirichlet process mixture models, whereas the conditional sampler is a variation of those recently introduced in the literature. For both the marginal and conditional methods, we consider as a running example a mixture model with an underlying normalized generalized Gamma process prior, and describe comparative simulation results demonstrating the efficacies of the proposed methods.

[1]  J. Kingman,et al.  Completely random measures. , 1967 .

[2]  W. Ewens The sampling theory of selectively neutral alleles. , 1972, Theoretical population biology.

[3]  T. Ferguson,et al.  A Representation of Independent Increment Processes without Gaussian Components , 1972 .

[4]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[5]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[6]  J. Kingman Random Discrete Distributions , 1975 .

[7]  D. Binder Bayesian cluster analysis , 1978 .

[8]  G. Shedler,et al.  Simulation of Nonhomogeneous Poisson Processes by Thinning , 1979 .

[9]  Yosihiko Ogata,et al.  On Lewis' simulation method for point processes , 1981, IEEE Trans. Inf. Theory.

[10]  École d'été de probabilités de Saint-Flour,et al.  École d'été de probabilités de Saint-Flour XIII - 1983 , 1985 .

[11]  D. Aldous Exchangeability and related topics , 1985 .

[12]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[13]  D. N. Geary Mixture Models: Inference and Applications to Clustering , 1989 .

[14]  N. Hjort Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[15]  W. Gilks,et al.  Adaptive Rejection Sampling for Gibbs Sampling , 1992 .

[16]  Radford M. Neal Bayesian Mixture Modeling , 1992 .

[17]  J. Pitman,et al.  Size-biased sampling of Poisson point processes and excursions , 1992 .

[18]  K. Roeder A Graphical Technique for Determining the Number of Components in a Mixture of Normals , 1994 .

[19]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[20]  M. Escobar Estimating Normal Means with a Dirichlet Process Prior , 1994 .

[21]  S. MacEachern Estimating normal means with a conjugate style dirichlet process prior , 1994 .

[22]  Walter R. Gilks,et al.  Hypothesis testing and model selection , 1995 .

[23]  P. Green Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[24]  Adrian E. Raftery,et al.  Hypothesis Testing and Model Selection Via Posterior Simulation , 1995 .

[25]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[26]  P. Müller,et al.  Bayesian curve fitting using multivariate normal mixtures , 1996 .

[27]  Adrian E. Raftery,et al.  Hypothesis testing and model selection , 1996 .

[28]  S. MacEachern,et al.  A semiparametric Bayesian model for randomised block designs , 1996 .

[29]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .

[30]  J. Pitman,et al.  The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator , 1997 .

[31]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[32]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[33]  M S Lewicki,et al.  A review of methods for spike sorting: the detection and classification of neural action potentials. , 1998, Network.

[34]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[35]  Steven N. MacEachern,et al.  Computational Methods for Mixture of Dirichlet Process Models , 1998 .

[36]  L. Tardella,et al.  Approximating distributions of random functionals of Ferguson‐Dirichlet priors , 1998 .

[37]  A. Brix Generalized Gamma measures and shot-noise Cox processes , 1999, Advances in Applied Probability.

[38]  M. Stephens Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[39]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[40]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[41]  P. Green,et al.  Modelling Heterogeneity With and Without the Dirichlet Process , 2001 .

[42]  Mario Medvedovic,et al.  Bayesian infinite mixture model based clustering of gene expression profiles , 2002, Bioinform..

[43]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[44]  Lancelot F. James,et al.  Poisson Process Partition Calculus with Applications to Exchangeable Models and Bayesian Nonparametrics , 2002 .

[45]  Lancelot F. James A simple proof of the almost sure discreteness of a class of random measures , 2003 .

[46]  A. Lijoi,et al.  Distributional results for means of normalized random measures with independent increments , 2003 .

[47]  J. Pitman Poisson-Kingman partitions , 2002, math/0210396.

[48]  Radford M. Neal Slice Sampling , 2003, The Annals of Statistics.

[49]  J. Pitman,et al.  Exchangeable Gibbs partitions and Stirling triangles , 2004, math/0412494.

[50]  Carl E. Rasmussen,et al.  Modelling Spikes with Mixtures of Factor Analysers , 2004, DAGM-Symposium.

[51]  Radford M. Neal,et al.  A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model , 2004 .

[52]  S. Walker,et al.  Normalized random measures driven by increasing additive processes , 2004, math/0508592.

[53]  Antonio Lijoi,et al.  Bayesian Nonparametric Analysis for a Generalized Dirichlet Process Prior , 2005 .

[54]  Ramsés H. Mena,et al.  Hierarchical Mixture Modeling With Normalized Inverse-Gaussian Priors , 2005 .

[55]  Ajay Jasra,et al.  Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling , 2005 .

[56]  D. B. Dahl Bayesian Inference for Gene Expression and Proteomics: Model-Based Clustering for Expression Data via a Dirichlet Process Mixture Model , 2006 .

[57]  J. Pitman Combinatorial Stochastic Processes , 2006 .

[58]  P. Müller,et al.  10 Model-Based Clustering for Expression Data via a Dirichlet Process Mixture Model , 2006 .

[59]  Lancelot F. James,et al.  Conjugacy as a Distinctive Feature of the Dirichlet Process , 2006 .

[60]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[61]  A. Fearnside Bayesian analysis of finite mixture distributions using the allocation sampler , 2007 .

[62]  Ramsés H. Mena,et al.  Controlling the reinforcement in Bayesian non‐parametric mixture models , 2007 .

[63]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[64]  Adrian E. Raftery,et al.  Bayesian Regularization for Normal Mixture Estimation and Model-Based Clustering , 2007, J. Classif..

[65]  P. Green,et al.  Bayesian Model-Based Clustering Procedures , 2007 .

[66]  S. Walker,et al.  Investigating nonparametric priors with Gibbs structure , 2008 .

[67]  Michael J. Black,et al.  A nonparametric Bayesian alternative to spike sorting , 2008, Journal of Neuroscience Methods.

[68]  Yee Whye Teh,et al.  Dependent Dirichlet Process Spike Sorting , 2008, NIPS.

[69]  O. Papaspiliopoulos A note on posterior sampling from Dirichlet mixture models , 2008 .

[70]  A. Lijoi,et al.  Models Beyond the Dirichlet Process , 2009 .

[71]  K. Ickstadt,et al.  Improved criteria for clustering based on the posterior similarity matrix , 2009 .

[72]  Zoubin Ghahramani,et al.  Modeling and Visualizing Uncertainty in Gene Expression Clusters Using Dirichlet Process Mixtures , 2009, TCBB.

[73]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[74]  Lancelot F. James,et al.  Posterior Analysis for Normalized Random Measures with Independent Increments , 2009 .

[75]  Lancelot F. James,et al.  On the posterior distribution of classes of random means , 2010, 1002.4276.

[76]  Luis E. Nieto-Barajas,et al.  A sensitivity analysis for Bayesian nonparametric density estimators , 2009 .

[77]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[78]  Thomas L. Griffiths,et al.  The Indian Buffet Process: An Introduction and Review , 2011, J. Mach. Learn. Res..

[79]  J. Griffin,et al.  Posterior Simulation of Normalized Random Measure Mixtures , 2011 .

[80]  R. Quiroga Spike sorting , 2012, Current Biology.

[81]  L. Trippa,et al.  A Class of Normalized Random Measures with an Exact Predictive Sampling Scheme , 2012 .

[82]  Michael I. Jordan,et al.  Cluster and Feature Modeling from Combinatorial Stochastic Processes , 2012, 1206.5862.

[83]  M. Steel,et al.  Comparing distributions by using dependent normalized random‐measure mixtures , 2013 .

[84]  Stephen G. Walker,et al.  Slice Sampling σ-Stable Poisson-Kingman Mixture Models , 2013 .

[85]  A. Lijoi,et al.  Modeling with normalized random measure mixture models , 2013, 1310.0260.

[86]  Tamara Broderick,et al.  Clusters and Features from Combinatorial Stochastic Processes , 2014 .