A constructive definition of the beta process

We derive a construction of the beta process that allows for the atoms with significant measure to be drawn first. Our representation is based on an extension of the Sethuraman (1994) construction of the Dirichlet process, and therefore we refer to it as a stick-breaking construction. Our first proof uses a limiting case argument of finite arrays. To this end, we present a finite sieve approximation to the beta process that parallels that of Ishwaran & Zarepour (2002) and prove its convergence to the beta process. We give a second proof of the construction using Poisson process machinery. We use the Poisson process to derive almost sure truncation bounds for the construction. We conclude the paper by presenting an efficient sampling algorithm for beta-Bernoulli and beta-negative binomial process models.

[1]  M. R. Leadbetter.  Poisson Processes , 2011, International Encyclopedia of Statistical Science.

[2]  Michael I. Jordan,et al.  Hierarchical Beta Processes and the Indian Buffet Process , 2007, AISTATS.

[3]  Lawrence Carin,et al.  Nonparametric factor analysis with beta process priors , 2009, ICML '09.

[4]  Michael I. Jordan,et al.  Beta Processes, Stick-Breaking and Power Laws , 2011, 1106.0539.

[5]  Yongdai Kim,et al.  On posterior consistency of survival models , 2001 .

[6]  E. Çinlar.  Probability and Stochastics , 2011 .

[7]  A. Cohen.  An Introduction to Probability Theory and Mathematical Statistics , 1979 .

[8]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[9]  Lawrence Carin,et al.  A Stick-Breaking Construction of the Beta Process , 2010, ICML.

[10]  D. Aldous.  Exchangeability and related topics , 1985 .

[11]  Vito Volterra,et al.  Theory of Functionals and of Integral and Integro-Differential Equations , 2005 .

[12]  Lawrence Carin,et al.  Negative Binomial Process Count and Mixture Modeling. , 2012, IEEE transactions on pattern analysis and machine intelligence.

[13]  Yongdai Kim.  NONPARAMETRIC BAYESIAN ESTIMATORS FOR COUNTING PROCESSES , 1999 .

[14]  Michael I. Jordan,et al.  Combinatorial Clustering and the Beta Negative Binomial Process , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[16]  Michael I. Jordan,et al.  Stick-Breaking Beta Processes and the Poisson Process , 2012, AISTATS.

[17]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[18]  Yongdai Kim,et al.  A new algorithm to generate beta processes , 2004, Comput. Stat. Data Anal..

[19]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[20]  Daniel M. Roy,et al.  The combinatorial structure of beta negative binomial processes , 2013, Bernoulli.

[21]  H. Ishwaran,et al.  DIRICHLET PRIOR SIEVES IN FINITE NORMAL MIXTURES , 2002 .

[22]  J. Kingman,et al.  Completely random measures. , 1967 .

[23]  Michael I. Jordan,et al.  Sharing Features among Dynamical Systems with Beta Processes , 2009, NIPS.

[24]  Chong Wang,et al.  The IBP Compound Dirichlet Process and its Application to Focused Topic Modeling , 2010, ICML.

[25]  Lawrence Carin,et al.  Variational Inference for Stick-Breaking Beta Process Priors , 2011, ICML.

[26]  N. Hjort.  Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[27]  Yee Whye Teh,et al.  Variational Inference for the Indian Buffet Process , 2009, AISTATS.

[28]  David B. Dunson,et al.  Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images , 2012, IEEE Transactions on Image Processing.

[29]  S. Walker,et al.  Beta-Stacy processes and a generalization of the Polya urn scheme , 1997 .