Bernoulli Society for Mathematical Statistics and Probability Version

We demonstrate how to calculate posteriors for general Bayesian nonparametric priors and likelihoods based on completely random measures (CRMs). We further show how to represent Bayesian nonparametric priors as a sequence of finite draws using a size-biasing approach—and how to represent full Bayesian nonparametric models via finite marginals. Motivated by conjugate priors based on exponential family representations of likelihoods, we introduce a notion of exponential families for CRMs, which we call exponential CRMs. This construction allows us to specify automatic Bayesian nonparametric conjugate priors for exponential CRM likelihoods. We demonstrate that our exponential CRMs allow particularly straightforward recipes for size-biased and marginal representations of Bayesian nonparametric models. Along the way, we prove that the gamma process is a conjugate prior for the Poisson likelihood process and the beta prime process is a conjugate prior for a process we call the odds Bernoulli process. We deliver a size-biased representation of the gamma process and a marginal representation of the gamma process coupled with a Poisson likelihood process.

[1]  Michael I. Jordan,et al.  Combinatorial Clustering and the Beta Negative Binomial Process , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Lancelot F. James Poisson Latent Feature Calculus for Generalized Indian Buffet Processes , 2014, 1411.2936.

[3]  Chong Wang,et al.  Variational inference in nonconjugate models , 2012, J. Mach. Learn. Res..

[4]  Michael I. Jordan,et al.  Cluster and Feature Modeling from Combinatorial Stochastic Processes , 2012, 1206.5862.

[5]  Michael I. Jordan,et al.  Stick-Breaking Beta Processes and the Poisson Process , 2012, AISTATS.

[6]  David B. Dunson,et al.  Beta-Negative Binomial Process and Poisson Factor Analysis , 2011, AISTATS.

[7]  Lawrence Carin,et al.  Variational Inference for Stick-Breaking Beta Process Priors , 2011, ICML.

[8]  Michael I. Jordan,et al.  Beta Processes, Stick-Breaking and Power Laws , 2011, 1106.0539.

[9]  Stephen G. Walker,et al.  Slice sampling mixture models , 2011, Stat. Comput..

[10]  Peter Orbanz,et al.  Conjugate Projective Limits , 2010, 1012.0363.

[11]  Lawrence Carin,et al.  A Stick-Breaking Construction of the Beta Process , 2010, ICML.

[12]  A. Lijoi,et al.  Models Beyond the Dirichlet Process , 2009 .

[13]  Y. Teh,et al.  Indian Buffet Processes with Power-law Behavior , 2009, NIPS.

[14]  Yee Whye Teh,et al.  Variational Inference for the Indian Buffet Process , 2009, AISTATS.

[15]  Lancelot F. James,et al.  Posterior Analysis for Normalized Random Measures with Independent Increments , 2009 .

[16]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[17]  Michael I. Jordan,et al.  Nonparametric bayesian models for machine learning , 2008 .

[18]  Michalis K. Titsias,et al.  The Infinite Gamma-Poisson Feature Model , 2007, NIPS.

[19]  Michael I. Jordan,et al.  Hierarchical Beta Processes and the Indian Buffet Process , 2007, AISTATS.

[20]  Yee Whye Teh,et al.  Stick-breaking Construction for the Indian Buffet Process , 2007, AISTATS.

[21]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[22]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[23]  Michael A. West,et al.  Hierarchical priors and mixture models, with applications in regression and density estimation , 2006 .

[24]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[25]  J. Pitman Poisson-Kingman partitions , 2002, math/0210396.

[26]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[27]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[28]  Radford M. Neal Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[29]  D. Dittmar Slice Sampling , 2000 .

[30]  Yongdai Kim NONPARAMETRIC BAYESIAN ESTIMATORS FOR COUNTING PROCESSES , 1999 .

[31]  P. Damlen,et al.  Gibbs sampling for Bayesian non‐conjugate and hierarchical models by using auxiliary variables , 1999 .

[32]  Michael A. West,et al.  Computing Nonparametric Hierarchical Models , 1998 .

[33]  J. Pitman Random discrete distributions invariant under size-biased permutation , 1996, Advances in Applied Probability.

[34]  J. Pitman Some developments of the Blackwell-MacQueen urn scheme , 1996 .

[35]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[36]  M. Escobar Estimating Normal Means with a Dirichlet Process Prior , 1994 .

[37]  S. MacEachern Estimating normal means with a conjugate style dirichlet process prior , 1994 .

[38]  J. Pitman,et al.  Size-biased sampling of Poisson point processes and excursions , 1992 .

[39]  J. Sethuraman A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[40]  N. Hjort Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[41]  Albert Y. Lo,et al.  On a Class of Bayesian Nonparametric Estimates: I. Density Estimates , 1984 .

[42]  Albert Y. Lo,et al.  Bayesian nonparametric statistical inference for Poisson point processes , 1982 .

[43]  P. Diaconis,et al.  Conjugate Priors for Exponential Families , 1979 .

[44]  T. Ferguson Prior Distributions on Spaces of Probability Measures , 1974 .

[45]  K. Doksum Tailfree and Neutral Random Probabilities and Their Posterior Distributions , 1974 .

[46]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[47]  M. Degroot Optimal Statistical Decisions , 1970 .

[48]  J. Kingman,et al.  Completely random measures. , 1967 .