Generalized Polya Urn for Time-varying Dirichlet Process Mixtures

Dirichlet Process Mixtures (DPMs) are a popular class of statistical models to perform density estimation and clustering. However, when the data available have a distribution evolving over time, such models are inadequate. We introduce here a class of time-varying DPMs which ensures that at each time step the random distribution follows a DPM model. Our model relies on an intuitive and simple generalized Polya urn scheme. Inference is performed using Markov chain Monte Carlo and Sequential Monte Carlo. We demonstrate our model on various applications.

[1]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[2]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[3]  Stephen G. Walker,et al.  Constructing First Order Stationary Autoregressive Models via Latent Processes , 2002 .

[4]  J. E. Griffin,et al.  Order-Based Dependent Dirichlet Processes , 2006 .

[5]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[6]  S. Roweis,et al.  Time-Varying Topic Models using Dependent Dirichlet Processes , 2005 .

[7]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.

[8]  P. Müller,et al.  A method for combining inference across related nonparametric Bayesian models , 2004 .

[9]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[10]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[11]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[12]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[13]  Stephen G. Walker,et al.  A bivariate Dirichlet process , 2003 .

[14]  Max Welling,et al.  Gibbs Sampling for (Coupled) Infinite Mixture Models in the Stick Breaking Representation , 2006, UAI.

[15]  N. Pillai,et al.  Bayesian density regression , 2007 .

[16]  J. Lafferty,et al.  Time-Sensitive Dirichlet Process Mixture Models , 2005 .

[17]  H. Joe.  Multivariate models and dependence concepts , 1998 .

[18]  Donato Michele Cifarelli,et al.  Nonparametric statistical problems under partial exchangeability . The role of associative means . Translated from Problemi statistici non parametrici in condizioni di scambiabilità parziale : impiego di medie associative , 2008 .

[19]  S. MacEachern,et al.  An ANOVA Model for Dependent Random Measures , 2004 .

[20]  J. Kingman.  Random partitions in population genetics , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[21]  S. MacEachern,et al.  Bayesian Nonparametric Spatial Modeling With Dirichlet Process Mixing , 2005 .