Bayesian Inference Via Classes of Normalized Random Measures

One of the main research areas in Bayesian Nonparametrics is the proposal and study of priors which generalize the Dirichlet process. Here we exploit theoretical properties of Poisson random measures in order to provide a comprehensive Bayesian analysis of random probabilities which are obtained by an appropriate normalization. Specifically we achieve explicit and tractable forms of the posterior and the marginal distributions, including an explicit and easily used description of generalizations of the important Blackwell-MacQueen Polya urn distribution. Such simplifications are achieved by the use of a latent variable which admits quite interesting interpretations which allow to gain a better understanding of the behaviour of these random probability measures. It is noteworthy that these models are generalizations of models considered by Kingman (1975) in a non-Bayesian context. Such models are known to play a significant role in a variety of applications including genetics, physics, and work involving random mappings and assemblies. Hence our analysis is of utility in those contexts as well. We also show how our results may be applied to Bayesian mixture models and describe computational schemes which are generalizations of known efficient methods for the case of the Dirichlet process. We illustrate new examples of processes which can play the role of priors for Bayesian nonparametric inference and finally point out some interesting connections with the theory of generalized gamma convolutions initiated by Thorin and further developed by Bondesson.

[1]  D. Freedman On the Asymptotic Behavior of Bayes' Estimates in the Discrete Case , 1963 .

[2]  J. McCloskey,et al.  A model for the distribution of individuals by species in an environment , 1965 .

[3]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[4]  W. Ewens The sampling theory of selectively neutral alleles. , 1972, Theoretical population biology.

[5]  T. Ferguson A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[6]  K. Doksum Tailfree and Neutral Random Probabilities and Their Posterior Distributions , 1974 .

[7]  Olof Thorin,et al.  On the infinite divisibility of the lognormal distribution , 1977 .

[8]  R. Kempton,et al.  Stochastic Abundance Models , 1980 .

[9]  Olof Thorin,et al.  An extension of the notion of a generalized Γ-convolution , 1978 .

[10]  Lennart Bondesson,et al.  A General Result on Infinite Divisibility , 1979 .

[11]  T. Rolski On random discrete distributions , 1980 .

[12]  B. Derrida Random-energy model: An exactly solvable model of disordered systems , 1981 .

[13]  D. Ruelle A mathematical reformulation of Derrida's REM and GREM , 1987 .

[14]  N. Hjort Nonparametric Bayes Estimators Based on Beta Processes in Models for Life History Data , 1990 .

[15]  J. Hansen FOR THE EWENS SAMPLING FORMULA , 1990 .

[16]  E. Regazzini,et al.  Distribution Functions of Means of a Dirichlet Process , 1990 .

[17]  Lennart Bondesson,et al.  Generalized Gamma Convolutions and Related Classes of Distributions and Densities , 1992 .

[18]  W. Sudderth,et al.  Polya Trees and Random Distributions , 1992 .

[19]  J. Pitman,et al.  Size-biased sampling of Poisson point processes and excursions , 1992 .

[20]  G. Grimmett,et al.  On the asymptotic distribution of large prime factors , 1993 .

[21]  M. Lavine More Aspects of Polya Tree Distributions for Statistical Modelling , 1992 .

[22]  M. Escobar Estimating Normal Means with a Dirichlet Process Prior , 1994 .

[23]  J. Pitman Exchangeable and partially exchangeable random partitions , 1995 .

[24]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[25]  J. Pitman Some developments of the Blackwell-MacQueen urn scheme , 1996 .

[26]  Jun S. Liu Nonparametric hierarchical Bayes via sequential imputations , 1996 .

[27]  J. Pitman,et al.  The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator , 1997 .

[28]  Jim Pitman,et al.  Partition structures derived from Brownian motion and stable subordinators , 1997 .

[29]  Yongdai Kim NONPARAMETRIC BAYESIAN ESTIMATORS FOR COUNTING PROCESSES , 1999 .

[30]  Simon Tavaré,et al.  The Poisson–Dirichlet Distribution and the Scale-Invariant Poisson Process , 1999, Combinatorics, Probability and Computing.

[31]  Eugenio Melilli,et al.  Some new results for dirichlet priors , 2000 .

[32]  N. Shephard,et al.  Non‐Gaussian Ornstein–Uhlenbeck‐based models and some of their uses in financial economics , 2001 .

[33]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[34]  Massimo Marinacci,et al.  APPLIED MATHEMATICS WORKING PAPER SERIES Risk, Ambiguity, and the Separation of Utility and Beliefs † , 2001 .

[35]  Svante Janson,et al.  Asymptotic distribution for the cost of linear probing hashing , 2001, Random Struct. Algorithms.

[36]  David E. Tyler,et al.  Regularity and uniqueness for constrained M-estimates and redescending M-estimates , 2001 .

[37]  T. Speed,et al.  Approximate Ewens formulae for symmetric overdominance selection , 2002 .

[38]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[39]  Lancelot F. James,et al.  Poisson Process Partition Calculus with Applications to Exchangeable Models and Bayesian Nonparametrics , 2002 .

[40]  F. Steutel,et al.  Infinite Divisibility of Probability Distributions on the Real Line , 2003 .

[41]  A. Lijoi,et al.  Distributional results for means of normalized random measures with independent increments , 2003 .

[42]  Domenico Menicucci,et al.  APPLIED MATHEMATICS WORKING PAPER SERIESOptimal Two-Object Auctions with Synergies * , 2001 .

[43]  J. Pitman Poisson-Kingman partitions , 2002, math/0210396.

[44]  R. Arratia,et al.  Logarithmic Combinatorial Structures: A Probabilistic Approach , 2003 .

[45]  D. R. Goldstein,et al.  Science and Statistics: A Festschrift for Terry Speed , 2003 .

[46]  I. Pruenster Random probability measures derived from increasing additive processes and their application to Bayesian statistics. , 2003 .

[47]  Lancelot F. James,et al.  Generalized weighted Chinese restaurant processes for species sampling mixture models , 2003 .

[48]  Lancelot F. James,et al.  Some further developments for stick-breaking priors: Finite and infinite clustering and classification , 2003 .

[50]  Means of a Dirichlet process and multiple hypergeometric functions , 2004, math/0410151.

[51]  S. Walker,et al.  Normalized random measures driven by increasing additive processes , 2004, math/0508592.

[52]  Lancelot F. James Functionals of dirichlet processes, the cifarelli-regazzini identity and beta-gamma processes , 2005, math/0505606.

[53]  Ramsés H. Mena,et al.  Hierarchical Mixture Modeling With Normalized Inverse-Gaussian Priors , 2005 .

[54]  Dudley Stark LOGARITHMIC COMBINATORIAL STRUCTURES: A PROBABILISTIC APPROACH (EMS Monographs in Mathematics) By R ICHARD A RRATIA , A. D. B ARBOUR and S IMON T AVARÉ : 363 pp., €69.00, ISBN 3-03719-000-0 (European Mathematical Society, 2003) , 2005 .

[55]  J. Pitman Combinatorial Stochastic Processes , 2006 .

[56]  Lancelot F. James Poisson calculus for spatial neutral to the right processes , 2003, math/0305053.