Large Sample Asymptotic for Nonparametric Mixture Model with Count Data

Bayesian nonparametric models have become popular recently due to its flexibility in identifying the unknown number of clusters. However, the flexibility comes at a cost for learning. Thus, Small Variance Asymptotic (SVA) is one of the promising approach for scalability in Bayesian nonparametric models. SVA approach for count data is also developed in which the likelihood function is replaced by the Kullback–Leibler divergence. In this paper, we present the Large Sample Asymptotic for count data when the number of sample in Multinomial distribution goes to infinity, we derive the similar result to SVA for scalable clustering.

[1]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[2]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[3]  Michael I. Jordan,et al.  Revisiting k-means: New Algorithms via Bayesian Nonparametrics , 2011, ICML.

[4]  Jonathan P. How,et al.  Dynamic Clustering via Asymptotics of the Dependent Dirichlet Process Mixture , 2013, NIPS.

[5]  Michael I. Jordan,et al.  Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models , 2012, NIPS.

[6]  Jun Zhu,et al.  DP-space: Bayesian Nonparametric Subspace Clustering with Small-variance Asymptotics , 2015, ICML.

[7]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[8]  Ke Jiang,et al.  Small-Variance Asymptotics for Hidden Markov Models , 2013, NIPS.

[9]  Seungjin Choi,et al.  Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility , 2015, AISTATS.

[10]  Jacques Dutka,et al.  The early history of the factorial function , 1991 .

[11]  Dinh Q. Phung,et al.  Bayesian Nonparametric Multilevel Clustering with Group-Level Contexts , 2014, ICML.

[12]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[13]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[14]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[15]  Jonathon Shlens,et al.  Notes on Kullback-Leibler Divergence and Likelihood , 2014, ArXiv.

[16]  Ardavan Saeedi,et al.  JUMP-Means: Small-Variance Asymptotics for Markov Jump Processes , 2015, ICML.

[17]  Jun Zhu,et al.  Small-Variance Asymptotics for Dirichlet Process Mixtures of SVMs , 2014, AAAI.

[18]  Jun S. Liu,et al.  The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation Problem , 1994 .