Spatial Normalized Gamma Processes

Dependent Dirichlet processes (DPs) are dependent sets of random measures, each being marginally DP distributed. They are used in Bayesian nonparametric models when the usual exchangeability assumption does not hold. We propose a simple and general framework to construct dependent DPs by marginalizing and normalizing a single gamma process over an extended space. The result is a set of DPs, each associated with a point in a space such that neighbouring DPs are more dependent. We describe Markov chain Monte Carlo inference involving Gibbs sampling and three different Metropolis-Hastings proposals to speed up convergence. We report an empirical study of convergence on a synthetic dataset and demonstrate an application of the model to topic modeling through time.

[1]  J. Kingman,et al.  Completely random measures. , 1967 .

[2]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[3]  S. MacEachern.  Decision Theoretic Aspects of Dependent Nonparametric Processes , 2000 .

[4]  Alan E. Gelfand,et al.  SPATIAL NONPARAMETRIC BAYESIAN MODELS , 2001 .

[5]  Carl E. Rasmussen,et al.  Infinite Mixtures of Gaussian Process Experts , 2001, NIPS.

[6]  Carl E. Rasmussen,et al.  Factorial Hidden Markov Models , 1997 .

[7]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[8]  Radford M. Neal,et al.  A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model , 2004 .

[9]  S. Walker,et al.  Normalized random measures driven by increasing additive processes , 2004, math/0508592.

[10]  Lancelot F. James,et al.  Bayesian Inference Via Classes of Normalized Random Measures , 2005, math/0503394.

[11]  S. MacEachern,et al.  Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing , 2005 .

[12]  S. Roweis,et al.  Time-Varying Topic Models using Dependent Dirichlet Processes , 2005 .

[13]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[14]  Thomas L. Griffiths,et al.  Learning Systems of Concepts with an Infinite Relational Model , 2006, AAAI.

[15]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[16]  Thomas L. Griffiths,et al.  Adaptor Grammars: A Framework for Specifying Compositional Nonparametric Bayesian Models , 2006, NIPS.

[17]  Jason A. Duan,et al.  Generalized spatial dirichlet process models , 2007 .

[18]  Arnaud Doucet,et al.  Generalized Polya Urn for Time-varying Dirichlet Process Mixtures , 2007, UAI.

[19]  E. Xing,et al.  Dynamic Non-Parametric Mixture Models and The Recurrent Chinese Restaurant Process a , 2008 .

[20]  L. Carin,et al.  The Matrix Stick-Breaking Process , 2008 .

[21]  A. Gelfand,et al.  The Nested Dirichlet Process , 2008 .

[22]  M. Steel,et al.  Bayesian nonparametric modelling with the Dirichlet process regression smoother , 2010 .

[23]  M. R. Leadbetter.  Poisson Processes , 2011, International Encyclopedia of Statistical Science.

[24]  J. Griffin.  The Ornstein–Uhlenbeck Dirichlet process and other time-varying processes for Bayesian nonparametric inference , 2011 .

[25]  D. Dunson,et al.  The local Dirichlet process , 2011, Annals of the Institute of Statistical Mathematics.