Bayesian Nonparametric Inference for the Power Likelihood

The aim in this article is to provide a means to undertake Bayesian inference for mixture models when the likelihood function is raised to a power between 0 and 1. The main purpose for doing this is to guarantee a strongly consistent model and hence, make it possible to compare the consistent posterior with the correct posterior, looking for signs of discrepancy. This will be explained in detail in the article. Another purpose would be for simulated annealing algorithms. In particular, for the widely used mixture of Dirichlet process model, it is far from obvious how to undertake inference via Markov chain Monte Carlo methods when the likelihood is raised to a power other than 1. In this article, we demonstrate how posterior sampling can be carried out when using a power likelihood. Matlab code to implement the algorithm is available as supplementary material.

[1]  S. Godsill.  On the Relationship Between Markov chain Monte Carlo Methods for Model Uncertainty , 2001 .

[2]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[3]  Radford M. Neal.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[4]  S. Walker.  New approaches to Bayesian consistency , 2004, math/0503672.

[5]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[6]  L. Wasserman,et al.  The consistency of posterior distributions in nonparametric problems , 1999 .

[7]  J. Ibrahim,et al.  Power prior distributions for regression models , 2000 .

[8]  J. Besag,et al.  Spatial Statistics and Bayesian Computation , 1993 .

[9]  A. Pettitt,et al.  Marginal likelihood estimation via power posteriors , 2008 .

[10]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[11]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[12]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[13]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[14]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[15]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[16]  N. Hjort,et al.  On Bayesian consistency , 2001 .

[17]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[18]  P. Damlen,et al.  Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables , 1999 .

[19]  J. Ghosh,et al.  POSTERIOR CONSISTENCY OF DIRICHLET MIXTURES IN DENSITY ESTIMATION , 1999 .