Phase Transition of Variational Bayes Learning in Bernoulli Mixture

The variational Bayes learning is widely used, because it provides both computational tractability and good generalization performance. However its theoretical properties, for example, the effect of hyperparameters to the learning result, has not yet been clarified. In this paper, we prove the asymptotic behavior of the variational free energy in a Bernoulli mixture model and show that the variational posterior distribution has phase transition according to the hyperparameters. We also discuss the design method of hyperparameters based on theoretical foundation.