As a member of the exponential family, the Dirichlet distribution has its conjugate prior. However, since the posterior distribution is difficult to use in practical problems, Bayesian estimation of the Dirichlet distribution, in general, is not analytically tractable. To derive practically easily used prior and posterior distributions, some approximations are required to approximate both the prior and the posterior distributions so that the conjugate match between the prior and posterior distributions holds and the obtained posterior distribution is easy to be employed. To this end, we approximate the distribution of the parameters in the Dirichlet distribution by a multivariate Gaussian distribution, based on the expectation propagation (EP) framework. The EP-based method captures the correlations among the parameters and provides an easily used prior/posterior distribution. Compared to recently proposed Bayesian estimation based on the variation inference (VI) framework, the EP-based method performs better with a smaller amount of observed data and is more stable.
[1]
Arne Leijon,et al.
Bayesian Estimation of Beta Mixture Models with Variational Inference
,
2011,
IEEE Transactions on Pattern Analysis and Machine Intelligence.
[2]
Michael I. Jordan,et al.
Probabilistic models of text and images
,
2004
.
[3]
Radford M. Neal.
Pattern Recognition and Machine Learning
,
2007,
Technometrics.
[4]
Zhanyu Ma.
Non-Gaussian Statistical Modelsand Their Applications
,
2011
.
[5]
Perry R. Cook,et al.
Bayesian Nonparametric Matrix Factorization for Recorded Music
,
2010,
ICML.
[6]
Manfred Opper,et al.
A Bayesian approach to on-line learning
,
1999
.
[7]
Arne Leijon,et al.
Vector quantization of LSF parameters with a mixture of dirichlet distributions
,
2013,
IEEE Transactions on Audio, Speech, and Language Processing.