Bayesian inference with dependent normalized completely random measures

The proposal and study of dependent prior processes has been a major research focus in the recent Bayesian nonparametric literature. In this paper, we introduce a flexible class of dependent nonparametric priors, investigate their properties and derive a suitable sampling scheme which allows their concrete implementation. The proposed class is obtained by normalizing dependent completely random measures, where the dependence arises by virtue of a suitable construction of the Poisson random measures underlying the completely random measures. We first provide general distributional results for the whole class of dependent completely random measures and then we specialize them to two specific priors, which represent the natural candidates for concrete implementation due to their analytic tractability: the bivariate Dirichlet and normalized $\sigma$-stable processes. Our analytical results, and in particular the partially exchangeable partition probability function, form also the basis for the determination of a Markov Chain Monte Carlo algorithm for drawing posterior inferences, which reduces to the well-known Blackwell--MacQueen P\'{o}lya urn scheme in the univariate case. Such an algorithm can be used for density estimation and for analyzing the clustering structure of the data and is illustrated through a real two-sample dataset example.

[1]  W. N. Bailey,et al.  Generalized hypergeometric series , 1935 .

[2]  I. S. Gradshteyn,et al.  Table of Integrals, Series, and Products , 1976 .

[3]  W. Ewens.  The sampling theory of selectively neutral alleles. , 1972, Theoretical population biology.

[4]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[5]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[6]  Robert C. Griffiths,et al.  A class of bivariate Poisson processes , 1978, Advances in Applied Probability.

[7]  S. MacEachern.  Estimating normal means with a conjugate style dirichlet process prior , 1994 .

[8]  J. Pitman.  Exchangeable and partially exchangeable random partitions , 1995 .

[9]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[10]  G. Constantine,et al.  A Multivariate Faa di Bruno Formula with Applications , 1996 .

[11]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[12]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[13]  Alan E. Gelfand,et al.  A Computational Approach for Full Nonparametric Bayesian Inference Under Dirichlet Process Mixture Models , 2002 .

[14]  A. Lijoi,et al.  Distributional results for means of normalized random measures with independent increments , 2003 .

[15]  I. Pruenster.  Random probability measures derived from increasing additive processes and their application to Bayesian statistics. , 2003 .

[16]  Ingram Olkin,et al.  A bivariate beta distribution , 2003 .

[17]  Fernando A. Quintana,et al.  Nonparametric Bayesian data analysis , 2004 .

[18]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[19]  Lancelot F. James.  Bayesian Poisson process partition calculus with an application to Bayesian Lévy moving averages , 2005, math/0508283.

[20]  Michael A. West,et al.  Hierarchical priors and mixture models, with applications in regression and density estimation , 2006 .

[21]  J. Pitman.  Combinatorial Stochastic Processes , 2006 .

[22]  Lancelot F. James,et al.  Conjugacy as a Distinctive Feature of the Dirichlet Process , 2006 .

[23]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[24]  Ramsés H. Mena,et al.  Controlling the reinforcement in Bayesian non‐parametric mixture models , 2007 .

[25]  G. Roberts,et al.  Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models , 2007, 0710.4228.

[26]  Michael I. Jordan,et al.  Hierarchical Bayesian Nonparametric Models with Applications , 2008 .

[27]  Ilenia Epifani,et al.  Nonparametric priors for vectors of survival functions , 2009 .

[28]  A. Lijoi,et al.  Models Beyond the Dirichlet Process , 2009 .

[29]  Lancelot F. James,et al.  Posterior Analysis for Normalized Random Measures with Independent Increments , 2009 .

[30]  Yee Whye Teh,et al.  Spatial Normalized Gamma Processes , 2009, NIPS.

[31]  David B. Dunson,et al.  Bayesian Nonparametrics: Nonparametric Bayes applications to biostatistics , 2010 .

[32]  Joseph Lipka,et al.  A Table of Integrals , 2010 .

[33]  Michael I. Jordan,et al.  Bayesian Nonparametrics: Hierarchical Bayesian nonparametric models with applications , 2010 .

[34]  Peter Orbanz,et al.  Projective Limit Random Probabilities on Polish Spaces , 2011, 1101.4657.

[35]  Fabrizio Leisen,et al.  Vectors of two-parameter Poisson-Dirichlet processes , 2011, J. Multivar. Anal..

[36]  Fernando A. Quintana,et al.  On the Support of MacEachern’s Dependent Dirichlet Processes and Extensions , 2012 .

[37]  Bernardo Nipoti.  Dependent Completely Random Measures and Statistical Applications , 2012 .

[38]  Jim E. Griffin,et al.  On Bayesian nonparametric modelling of two correlated distributions , 2013, Stat. Comput..

[39]  Ulrike Goldschmidt,et al.  An Introduction To The Theory Of Point Processes , 2016 .

[40]  D. Blei.  Bayesian Nonparametrics I , 2016 .

[41]  Kerstin Vogler,et al.  Table Of Integrals Series And Products , 2016 .