论文信息 - Variational Bayesian Stochastic Complexity of Mixture Models

Variational Bayesian Stochastic Complexity of Mixture Models

The Variational Bayesian framework has been widely used to approximate the Bayesian learning. In various applications, it has provided computational tractability and good generalization performance. In this paper, we discuss the Variational Bayesian learning of the mixture of exponential families and provide some additional theoretical support by deriving the asymptotic form of the stochastic complexity. The stochastic complexity, which corresponds to the minimum free energy and a lower bound of the marginal likelihood, is a key quantity for model selection. It also enables us to discuss the effect of hyperparameters and the accuracy of the Variational Bayesian approach as an approximation of the true Bayesian learning.

Kazuho Watanabe | Sumio Watanabe

[1] Masa-aki Sato,et al. Online Model Selection Based on the Variational Bayes , 2001, Neural Computation.

[2] J. Hartigan. A failure of likelihood asymptotics for normal mixtures , 1985 .

[3] Matthew J. Beal,et al. Graphical Models and Variational Methods , 2001 .

[4] Sumio Watanabe,et al. Algebraic Analysis for Nonidentifiable Learning Machines , 2001, Neural Computation.

[5] David J. C. MacKay,et al. Bayesian Interpolation , 1992, Neural Computation.

[6] L. Brown. Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[7] Kazuho Watanabe,et al. Stochastic Complexity for Mixture of Exponential Families in Variational Bayes , 2005, ALT.

[8] Sumio Watanabe,et al. Singularities in mixture models and upper bounds of stochastic complexity , 2003, Neural Networks.

[9] G. Schwarz. Estimating the Dimension of a Model , 1978 .

[10] Lawrence D. Brown. Fundamentals of Statistical Exponential Families , 1987 .

[11] Sumio Watanabe. Algebraic Analysis for Non-identifiable Learning Machines , 2000 .

[12] Hirotugu Akaike,et al. Likelihood and the Bayes procedure , 1980 .

[13] K. Watanabe,et al. Lower bounds of stochastic complexities in variational Bayes learning of Gaussian mixture models , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[14] Hagai Attias,et al. Inferring Parameters and Structure of Latent Variable Models by Variational Bayes , 1999, UAI.