Mean-field variational approximate Bayesian inference for latent variable models

The ill-posed nature of missing variable models offers a challenging testing ground for new computational techniques. This is the case for the mean-field variational Bayesian inference. The behavior of this approach in the setting of the Bayesian probit model is illustrated. It is shown that the mean-field variational method always underestimates the posterior variance and, that, for small sample sizes, the mean-field variational approximation to the posterior location could be poor.

[1]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[2]  Wim Wiegerinck,et al.  Variational Approximations between Mean Field Theory and the Junction Tree Algorithm , 2000, UAI.

[3]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[4]  Zoubin Ghahramani,et al.  Propagation Algorithms for Variational Bayesian Learning , 2000, NIPS.

[5]  Bo Wang,et al.  Convergence and Asymptotic Normality of Variational Bayesian Approximations for Expon , 2004, UAI.

[6]  Michael I. Jordan,et al.  Bayesian parameter estimation via variational methods , 2000, Stat. Comput..

[7]  Steffen L. Lauritzen.  Highly Structured Stochastic Systems , 2003 .

[8]  Tommi S. Jaakkola,et al.  Tutorial on variational approximation methods , 2000 .

[9]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[10]  D. Titterington,et al.  Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model , 2006 .

[11]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[12]  Michael I. Jordan.  Graphical Models , 2003 .

[13]  Steffen L. Lauritzen,et al.  Some modern applications of graphical models , 2003 .

[14]  J. G. Bethlehem,et al.  COMPSTAT 2000, Proceedings in Computational Statistics , 2000 .

[15]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[16]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[17]  Brian Everitt,et al.  An Introduction to Latent Variable Models , 1984 .

[18]  Kevin Humphreys,et al.  Some examples of recursive variational approximations for Bayesian inference , 2001 .

[19]  Matthew J. Beal,et al.  The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures , 2003 .

[20]  M. Opper,et al.  Some Examples of Recursive Variational Approximations for Bayesian Inference , 2001 .

[21]  D. Titterington,et al.  Approximate Bayesian inference for simple mixtures , 2000 .

[22]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[23]  G. W. Snedecor.  Statistical Methods , 1964 .

[24]  M. Opper,et al.  Advanced mean field methods: theory and practice , 2001 .

[25]  Adrian E. Raftery,et al.  Bayes factors and model uncertainty , 1995 .

[26]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[27]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[28]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.