Bayesian Exponential Family Harmoniums

A Bayesian Exponential Family Harmonium (BEFH) model is pre sented for topical modeling of text and multimedia data, and for “posterior” latent semant ic projection of such data for subsequent data mining tasks. BEFHs are a Bayesian approach to inferenc a d learning with the recently proposed EFH models and their variants, which enables smoot hed, robust estimation of the topicattribute coupling coefficients that are reminiscent of the smoothed topical word-probabilities in the latent Dirichlet Allocation (LDA) model. The Langevin a lgorithm conjoint with an MCMC scheme is applied for posterior inference with BEFH. An empi rical Bayes method is also developed to estimate the hyperparameters.

[1]  Christopher Joseph Pal,et al.  Multi-Conditional Learning: Generative/Discriminative Training for Clustering and Classification , 2006, AAAI.

[2]  Peter V. Gehler,et al.  The rate adapting poisson model for information retrieval and object recognition , 2006, ICML.

[3]  Rong Yan,et al.  Mining Associated Text and Images with Dual-Wing Harmoniums , 2005, UAI.

[4]  Yuan Qi,et al.  Diagram structure recognition by Bayesian conditional random fields , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Miguel Á. Carreira-Perpiñán,et al.  On Contrastive Divergence Learning , 2005, AISTATS.

[6]  Yuan Qi,et al.  Bayesian Conditional Random Fields , 2005, AISTATS.

[7]  Geoffrey E. Hinton,et al.  Exponential Family Harmoniums with an Application to Information Retrieval , 2004, NIPS.

[8]  Zoubin Ghahramani,et al.  Bayesian Learning in Undirected Graphical Models: Approximate MCMC Algorithms , 2004, UAI.

[9]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[10]  Paul Over,et al.  TRECVID: Benchmarking the Effectivenss of Information Retrieval Tasks on Digital Video , 2003, CIVR.

[11]  David A. Forsyth,et al.  Matching Words and Pictures , 2003, J. Mach. Learn. Res..

[12]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[13]  Michael I. Jordan,et al.  1 Matching Words and Pictures , 2003 .

[14]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[15]  A. Childs,et al.  Exact sampling from nonattractive distributions using summary states. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  Thomas Hofmann,et al.  Probabilistic latent semantic indexing , 1999, SIGIR '99.

[17]  J. Sexton,et al.  Hamiltonian evolution for the hybrid Monte Carlo algorithm , 1992 .

[18]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..