Practical collapsed variational bayes inference for hierarchical dirichlet process

We propose a novel collapsed variational Bayes (CVB) inference for the hierarchical Dirichlet process (HDP). While the existing CVB inference for the HDP variant of latent Dirichlet allocation (LDA) is more complicated and harder to implement than that for LDA, the proposed algorithm is simple to implement, does not require variance counts to be maintained, does not need to set hyper-parameters, and has good predictive performance.

[1]  Chong Wang,et al.  Collaborative topic modeling for recommending scientific articles , 2011, KDD.

[2]  Francis R. Bach,et al.  Online Learning for Latent Dirichlet Allocation , 2010, NIPS.

[3]  David M. Blei,et al.  Connections between the lines: augmenting social networks with text , 2009, KDD.

[4]  A. Asuncion.  Approximate Mean Field for Dirichlet-Based Models , 2010 .

[5]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[6]  Chong Wang,et al.  Online Variational Inference for the Hierarchical Dirichlet Process , 2011, AISTATS.

[7]  Brian D. Davison,et al.  Tracking trends: incorporating term volume into temporal topic models , 2011, KDD.

[8]  Lancelot F. James,et al.  Gibbs Sampling Methods for Stick-Breaking Priors , 2001 .

[9]  Thomas L. Griffiths,et al.  Probabilistic author-topic models for information discovery , 2004, KDD.

[10]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[11]  Bo Zhao,et al.  Probabilistic topic models with biased propagation on heterogeneous information networks , 2011, KDD.

[12]  Ning Chen,et al.  Conditional topical coding: an efficient topic model conditioned on rich features , 2011, KDD.

[13]  Yee Whye Teh,et al.  Collapsed Variational Inference for HDP , 2007, NIPS.

[14]  Daniel M. Roy,et al.  Complexity of Inference in Latent Dirichlet Allocation , 2011, NIPS.

[15]  Christos Faloutsos,et al.  Graphs over time: densification laws, shrinking diameters and possible explanations , 2005, KDD '05.

[16]  Thomas L. Griffiths,et al.  The Author-Topic Model for Authors and Documents , 2004, UAI.

[17]  Ramesh Nallapati,et al.  Joint latent topic models for text and citations , 2008, KDD.

[18]  Dafna Shahaf,et al.  Turning down the noise in the blogosphere , 2009, KDD.

[19]  Yee Whye Teh,et al.  On Smoothing and Inference for Topic Models , 2009, UAI.

[20]  Susan T. Dumais,et al.  Partially labeled topic models for interpretable text mining , 2011, KDD.

[21]  Hiroshi Nakagawa,et al.  Deterministic Single-Pass Algorithm for LDA , 2010, NIPS.

[22]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[23]  Andrew McCallum,et al.  Efficient methods for topic model inference on streaming document collections , 2009, KDD.

[24]  Padhraic Smyth,et al.  Statistical entity-topic models , 2006, KDD '06.

[25]  Eric P. Xing,et al.  Structured correspondence topic models for mining captioned figures in biological literature , 2009, KDD.

[26]  Yee Whye Teh,et al.  A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation , 2006, NIPS.

[27]  T. Minka.  Estimating a Dirichlet distribution , 2012 .

[28]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[29]  Sean Gerrish,et al.  Predicting Legislative Roll Calls from Text , 2011, ICML.

[30]  Michal Rosen-Zvi,et al.  Latent Topic Models for Hypertext , 2008, UAI.

[31]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[32]  Charles Elkan,et al.  Clustering documents with an exponential-family approximation of the Dirichlet compound multinomial distribution , 2006, ICML.

[33]  Andrew McCallum,et al.  Rethinking LDA: Why Priors Matter , 2009, NIPS.

[34]  Hiroshi Nakagawa,et al.  Topic models with power-law using Pitman-Yor process , 2010, KDD.

[35]  Sean Gerrish,et al.  A Language-based Approach to Measuring Scholarly Impact , 2010, ICML.

[36]  John D. Lafferty,et al.  Dynamic topic models , 2006, ICML.

[37]  W. Bruce Croft,et al.  LDA-based document models for ad-hoc retrieval , 2006, SIGIR.

[38]  Chong Wang,et al.  Continuous Time Dynamic Topic Models , 2008, UAI.

[39]  Yasushi Sakurai,et al.  Online multiscale dynamic topic models , 2010, KDD.

[40]  David Buttler,et al.  Latent topic feedback for information retrieval , 2011, KDD.