Enriched Pitman-Yor processes

In Bayesian nonparametrics there exists a rich variety of discrete priors, including the Dirichlet process and its generalizations, which are nowadays well-established tools. Despite the remarkable advances, few proposals are tailored for modeling observations lying on product spaces, such as $\mathbb{R}^p$. In this setting, most of the available priors lack of flexibility and they do not allow for separate partition structures among the spaces. We address these issues by introducing a discrete nonparametric prior termed enriched Pitman-Yor process (EPY). Theoretical properties of this novel prior are extensively investigated. Specifically, we discuss its formal link with the enriched Dirichlet process and normalized random measures, we describe a square-breaking representation and we obtain closed form expressions for the posterior law and the involved urn schemes. In second place, we show that several existing approaches, including Dirichlet processes with a spike and slab base measure and mixture of mixtures models, implicitly rely on special cases of the EPY, which therefore constitutes a unified probabilistic framework for many Bayesian nonparametric priors. Interestingly, our unifying formulation will allow to naturally extend these models, while preserving their analytical tractability. As an illustration, we employ the EPY for a species sampling problem in ecology.

[1]  Tommaso Rigon,et al.  An enriched mixture model for functional clustering , 2019, Applied Stochastic Models in Business and Industry.

[2]  Lancelot F. James Functionals of dirichlet processes, the cifarelli-regazzini identity and beta-gamma processes , 2005, math/0505606.

[3]  David B. Dunson,et al.  Bayesian Methods for Highly Correlated Exposure Data , 2007, Epidemiology.

[4]  T. Ferguson,et al.  A Representation of Independent Increment Processes without Gaussian Components , 1972 .

[5]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[6]  A. Lijoi,et al.  Distributional results for means of normalized random measures with independent increments , 2003 .

[7]  David B. Dunson,et al.  Improving prediction from dirichlet process mixtures via enrichment , 2014, J. Mach. Learn. Res..

[8]  Matteo Ruggiero,et al.  Are Gibbs-Type Priors the Most Natural Generalization of the Dirichlet Process? , 2015, IEEE transactions on pattern analysis and machine intelligence.

[9]  Sonia Petrone,et al.  An enriched conjugate prior for Bayesian nonparametric inference , 2011 .

[10]  A. Lijoi,et al.  Bayesian inference with dependent normalized completely random measures , 2014, 1407.0482.

[11]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[12]  Ramsés H. Mena,et al.  Controlling the reinforcement in Bayesian non‐parametric mixture models , 2007 .

[13]  Guido Consonni,et al.  Conditionally Reducible Natural Exponential Families and Enriched Conjugate Priors , 2001 .

[14]  Means of a Dirichlet process and multiple hypergeometric functions , 2004, math/0410151.

[15]  Aad van der Vaart,et al.  Fundamentals of Nonparametric Bayesian Inference , 2017 .

[16]  Ramsés H. Mena,et al.  Bayesian Nonparametric Estimation of the Probability of Discovering New Species , 2007 .

[17]  Alexis Boukouvalas,et al.  Enriched Mixtures of Gaussian Process Experts , 2019, ArXiv.

[18]  A. Lijoi,et al.  On the Pitman–Yor process with spike and slab base measure , 2017 .

[19]  Bruno Scarpa,et al.  Bayesian Hierarchical Functional Data Analysis Via Contaminated Informative Priors , 2009, Biometrics.

[20]  A. Lijoi,et al.  Models Beyond the Dirichlet Process , 2009 .

[21]  E. Regazzini,et al.  Distribution Functions of Means of a Dirichlet Process , 1990 .

[22]  L. Carin,et al.  The Matrix Stick-Breaking Process , 2008 .

[23]  Gertraud Malsiner-Walli,et al.  Identifying Mixtures of Mixtures Using Bayesian Estimation , 2015, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[24]  J. Pitman,et al.  Size-biased sampling of Poisson point processes and excursions , 1992 .

[25]  Marina Vannucci,et al.  Spiked Dirichlet Process Prior for Bayesian Multiple Hypothesis Testing in Random Effects Models. , 2009, Bayesian analysis.

[26]  Federico Camerlenghi,et al.  Distribution theory for hierarchical processes , 2019, The Annals of Statistics.

[27]  Bruno Scarpa,et al.  Enriched Stick-Breaking Processes for Functional Data , 2014, Journal of the American Statistical Association.

[28]  J. Pitman Some developments of the Blackwell-MacQueen urn scheme , 1996 .

[29]  D. Dunson,et al.  Bayesian Selection and Clustering of Polymorphisms in Functionally Related Genes , 2008 .

[30]  Ramsés H. Mena,et al.  Bayesian non‐parametric inference for species variety with a two‐parameter Poisson–Dirichlet process prior , 2009 .

[31]  M. Vannucci,et al.  A Bayesian Nonparametric Spiked Process Prior for Dynamic Model Selection , 2019, Bayesian Analysis.

[32]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[33]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[34]  J. Terborgh,et al.  Hyperdominance in the Amazonian Tree Flora , 2013, Science.

[35]  Francesco Denti Bayesian mixtures for large scale inference , 2020 .

[36]  J. Pitman,et al.  The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator , 1997 .

[37]  S. Kerov,et al.  The Markov–Krein Correspondence in Several Dimensions , 2004 .

[38]  P. Müller,et al.  A Bayesian discovery procedure , 2009, Journal of the Royal Statistical Society. Series B, Statistical methodology.