On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)

New methodology for fully Bayesian mixture analysis is developed, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture. A sample from the full joint distribution of all unknown variables is thereby generated, and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution. The methodology is applied here to the analysis of univariate normal mixtures, using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  G. Box.  An analysis of transformations (with discussion) , 1964 .

[3]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[4]  R. Elston,et al.  A general model for the genetic analysis of pedigree data. , 1971, Human heredity.

[5]  P. Peskun,et al.  Optimum Monte-Carlo sampling using Markov chains , 1973 .

[6]  N E Morton,et al.  Skewness in commingled distributions. , 1976, Biometrics.

[7]  D. Binder.  Bayesian cluster analysis , 1978 .

[8]  N. Laird.  Nonparametric Maximum Likelihood Estimation of a Mixing Distribution , 1978 .

[9]  Probability functions on complex pedigrees , 1978 .

[10]  B. Lindsay.  The Geometry of Mixture Likelihoods: A General Theory , 1983 .

[11]  Philip Heidelberger,et al.  Simulation Run Length Control in the Presence of an Initial Transient , 1983, Oper. Res..

[12]  Dankmar Böhning,et al.  Numerical estimation of a probability measure , 1985 .

[13]  S. Weisberg.  Plots, transformations, and regression , 1985 .

[14]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[15]  G. McLachlan.  On Bootstrapping the Likelihood Ratio Test Statistic for the Number of Components in a Normal Mixture , 1987 .

[16]  Haluk Derin,et al.  Modeling and Segmentation of Noisy and Textured Images Using Gibbs Random Fields , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[18]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[19]  N. Mendell,et al.  Simulated percentage points for the null distribution of the likelihood ratio test for a mixture of two normals. , 1988, Biometrics.

[20]  A. Izenman,et al.  Philatelic Mixtures and Multimodal Densities , 1988 .

[21]  References to discussion , 1988 .

[22]  I. Pitas.  Markovian image models for image labeling and edge detection , 1988 .

[23]  J. Besag,et al.  Generalized Monte Carlo significance tests , 1989 .

[24]  Brandt,et al.  Simulations without critical slowing down: Ising and three-state Potts models. , 1989, Physical review. B, Condensed matter.

[25]  H. Tong.  Non-linear time series. A dynamical system approach , 1990 .

[26]  K. Roeder.  Density estimation with confidence sets exemplified by superclusters and voids in the galaxies , 1990 .

[27]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .

[28]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[29]  Adele Cutler,et al.  Information Ratios for Validating Mixture Analysis , 1992 .

[30]  M. Degroot,et al.  Modeling lake-chemistry distributions: approximate Bayesian methods for estimating a finite-mixture model , 1992 .

[31]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[32]  M. Tanner,et al.  Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler , 1992 .

[33]  M. West,et al.  A Bayesian method for classification and discrimination , 1992 .

[34]  A. Baddeley,et al.  Stochastic geometry models in high-level vision , 1993 .

[35]  J Besag,et al.  DISCUSSION ON THE MEETING ON THE GIBBS SAMPLER AND OTHER MARKOV CHAIN-MONTE CARLO METHODS , 1993 .

[36]  Thomas H. Short.  An algorithm for the detection and measurement of rail surface defects , 1993 .

[37]  Michael I. Miller,et al.  REPRESENTATIONS OF KNOWLEDGE IN COMPLEX SYSTEMS , 1994 .

[38]  Z. D. Feng,et al.  Using Bootstrap Likelihood Ratio in Finite Mixture Models , 1994 .

[39]  C. Geyer,et al.  Simulation Procedures and Likelihood Inference for Spatial Point Processes , 1994 .

[40]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[41]  V. Johnson.  A Model for Segmentation and Analysis of Noisy Images , 1994 .

[42]  S. Crawford.  An Application of the Laplace Method to Finite Mixture Distributions , 1994 .

[43]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[44]  Anthony C. Atkinson,et al.  Multivariate Transformations, Regression Diagnostics and Seemingly Unrelated Regression , 1995 .

[45]  J. Besag,et al.  Bayesian Computation and Stochastic Systems , 1995 .

[46]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[47]  A. O'Hagan,et al.  Fractional Bayes factors for model comparison , 1995 .

[48]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[49]  P. Green,et al.  Bayesian Computation and Stochastic , 1995 .

[50]  C. Robert.  Convergence Control Methods for Markov Chain Monte Carlo Algorithms , 1995 .

[51]  Gilles Celeux,et al.  On Stochastic Versions of the EM Algorithm , 1995 .

[52]  Walter R. Gilks,et al.  Bayesian model comparison via jump diffusions , 1995 .

[53]  D. Böhning.  A review of reliable maximum likelihood algorithms for semiparametric mixture models , 1995 .

[54]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[55]  B. Lindsay.  Mixture models : theory, geometry, and applications , 1995 .

[56]  G. Celeux,et al.  Stochastic versions of the em algorithm: an experimental study in the mixture case , 1996 .

[57]  C. Robert.  Mixtures of Distributions: Inference and Estimation , 1996 .

[58]  P Schlattmann,et al.  Covariate adjusted mixture models and disease mapping with the program DismapWin. , 1996, Statistics in medicine.

[59]  P. Müller,et al.  Bayesian curve fitting using multivariate normal mixtures , 1996 .

[60]  J. Berger,et al.  The Intrinsic Bayes Factor for Model Selection and Prediction , 1996 .

[61]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[62]  Joong-Kweon Sohn,et al.  Convergence Diagnostics for the Gibbs Sampler , 1996 .

[63]  Adrian E. Raftery,et al.  Hypothesis testing and model selection , 1996 .

[64]  J. Bezdek,et al.  Estimating the Number of Components in a Normal Mixture , 1996 .

[65]  M. Stephens.  Dealing with the Multimodal Distributions of Mixture Model Parameters , 1996 .

[66]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[67]  Marco Bernardo,et al.  Noninformative Priors Do Not Exist: A Discussion with Jos , 1997 .

[68]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .

[69]  S Richardson,et al.  A Bayesian approach to multipoint mapping in nuclear families , 1997, Genetic epidemiology.

[70]  Michael A. West,et al.  Hierarchical Mixture Models in Neurological Transmission Analysis , 1997 .

[71]  N. Cressie,et al.  Object Identification Using Markov Random Field Segmentation Models at Multiple Resolutions of a Rectangular Lattice , 1997 .

[72]  A. Sokal.  Monte Carlo Methods in Statistical Mechanics: Foundations and New Algorithms , 1997 .

[73]  S. Heath.  Markov chain Monte Carlo segregation and linkage analysis for oligogenic models. , 1997, American journal of human genetics.

[74]  Xiao-Li Meng,et al.  The EM Algorithm—an Old Folk‐song Sung to a Fast New Tune , 1997 .

[75]  A. Raftery,et al.  Nearest-Neighbor Clutter Removal for Estimating Features in Spatial Point Processes , 1998 .

[76]  A. Raftery,et al.  Detecting features in spatial point processes with clutter via model-based clustering , 1998 .

[77]  D. Böhning,et al.  Some News about C.A.MAN Computer Assisted Analysis of Mixtures , 1998 .

[78]  Brani Vidakovic,et al.  Bayesian Inference with Wavelets: Density Estimation , 1998 .

[79]  M. Aitkin.  A General Maximum Likelihood Analysis of Variance Components in Generalized Linear Models , 1999, Biometrics.