Univariate Bayesian nonparametric mixture modeling with unimodal kernels

Within the context of mixture modeling, the normal distribution is typically used as the components distribution. However, if a cluster is skewed or heavy tailed, then the normal distribution will be inefficient and many may be needed to model a single cluster. In this paper, we present an attempt to solve this problem. We define a cluster, in the absence of further information, to be a group of data which can be modeled by a unimodal density function.Hence, our intention is to use a family of univariate distribution functions, to replace the normal, for which the only constraint is unimodality. With this aim, we devise a new family of nonparametric unimodal distributions, which has large support over the space of univariate unimodal distributions.The difficult aspect of the Bayesian model is to construct a suitable MCMC algorithm to sample from the correct posterior distribution. The key will be the introduction of strategic latent variables and the use of the Product Space view of Reversible Jump methodology.

[1]  S. Walker,et al.  Sampling Truncated Normal, Beta, and Gamma Densities , 2001 .

[2]  M. Postman,et al.  Probes of large-scale structure in the Corona Borealis region. , 1986 .

[3]  Agostino Nobile,et al.  Bayesian finite mixtures with an unknown number of components: The allocation sampler , 2007, Stat. Comput..

[4]  Michael P. Wiper,et al.  Mixtures of Gamma Distributions With Applications , 2001 .

[5]  M. Stephens.  Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods , 2000 .

[6]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[7]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[8]  Albert Y. Lo,et al.  Bayes Methods for a Symmetric Unimodal Density and its Mode , 1989 .

[9]  D. Blackwell.  Discreteness of Ferguson Selections , 1973 .

[10]  C. Robert,et al.  Estimation of Finite Mixture Distributions Through Bayesian Sampling , 1994 .

[11]  S. Godsill.  On the Relationship Between Markov chain Monte Carlo Methods for Model Uncertainty , 2001 .

[12]  W. Rudin.  Principles of mathematical analysis , 1964 .

[13]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[14]  Arnost Komárek,et al.  A new R package for Bayesian estimation of multivariate normal mixtures allowing for selection of the number of components and interval-censored data , 2009, Comput. Stat. Data Anal..

[15]  Tao Dai.  On multivariate unimodal distributions , 1989 .

[16]  M. Steel,et al.  On Bayesian Modelling of Fat Tails and Skewness , 1998 .

[17]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[18]  Albert Y. Lo,et al.  On a Class of Bayesian Nonparametric Estimates: I. Density Estimates , 1984 .

[19]  T. Ferguson.  BAYESIAN DENSITY ESTIMATION BY MIXTURES OF NORMAL DISTRIBUTIONS , 1983 .

[20]  R. Rigby,et al.  Generalized additive models for location, scale and shape , 2005 .

[21]  Scott A. Sisson,et al.  Transdimensional Markov Chains , 2005 .

[22]  Mark F. J. Steel,et al.  Flexible univariate continuous distributions , 2009 .

[23]  A. Gelfand,et al.  Bayesian Semiparametric Median Regression Modeling , 2001 .

[24]  M. Stephens.  Dealing with label switching in mixture models , 2000 .

[25]  Brian Gough,et al.  GNU Scientific Library Reference Manual - Third Edition , 2003 .

[26]  George Iliopoulos,et al.  An Artificial Allocations Based Solution to the Label Switching Problem in Bayesian Analysis of Mixtures of Distributions , 2010 .

[27]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[28]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[29]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[30]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[31]  Stephen G. Walker,et al.  Label Switching in Bayesian Mixture Models: Deterministic Relabeling Strategies , 2014 .

[32]  Ajay Jasra,et al.  Markov Chain Monte Carlo Methods and the Label Switching Problem in Bayesian Mixture Modeling , 2005 .

[33]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[34]  P. Green,et al.  Trans-dimensional Markov chain Monte Carlo , 2000 .

[35]  Sylvia Frühwirth-Schnatter,et al.  Finite Mixture and Markov Switching Models , 2006 .

[36]  Luc Devroye,et al.  Random variate generation for multivariate unimodal densities , 1997, TOMC.

[37]  P. Green,et al.  On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion) , 1997 .

[38]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[39]  M. Escobar.  Estimating Normal Means with a Dirichlet Process Prior , 1994 .