The variational approximation for Bayesian inference

The influence of this Thomas Bayes' work was immense. It was from here that "Bayesian" ideas first spread through the mathematical world, as Bayes's own article was ignored until 1780 and played no important role in scientific debate until the 20th century. It was also this article of Laplace's that introduced the mathematical techniques for the asymptotic analysis of posterior distributions that are still employed today. And it was here that the earliest example of optimum estimation can be found, the derivation and characterization of an estimator that minimized a particular measure of posterior expected loss. After more than two centuries, we mathematicians, statisticians cannot only recognize our roots in this masterpiece of our science, we can still learn from it.

[1]  Zhengrong Liang,et al.  Bayesian reconstruction in emission computerized tomography , 1988 .

[2]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[3]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[4]  ModelsbyTommi S. Jaakkola.  Variational Methods for Inference and Estimation inGraphical , 1997 .

[5]  L. G. Byars,et al.  Positron emission tomographic images and expectation maximization: a VLSI architecture for multiple iterations per second , 1988 .

[6]  Stephen M. Stigler,et al.  Thomas Bayes's Bayesian Inference , 1982 .

[7]  Nikolas P. Galatsanos,et al.  Super-Resolution Based on Fast Registration and Maximum a Posteriori Reconstruction , 2007, IEEE Transactions on Image Processing.

[8]  Aggelos K. Katsaggelos,et al.  Total Variation Image Restoration and Parameter Estimation using Variational Posterior Distribution Approximation , 2007, 2007 IEEE International Conference on Image Processing.

[9]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[10]  R. Bellman.  Calculus of Variations (L. E. Elsgolc) , 1963 .

[11]  Adrian Corduneanu,et al.  Variational Bayesian Model Selection for Mixture Distributions , 2001 .

[12]  T. Moon.  The expectation-maximization algorithm , 1996, IEEE Signal Process. Mag..

[13]  William D. Penny,et al.  Variational Bayes for generalized autoregressive models , 2002, IEEE Trans. Signal Process..

[14]  David J. C. MacKay,et al.  Variational Gaussian process classifiers , 2000, IEEE Trans. Neural Networks Learn. Syst..

[15]  Nikolas P. Galatsanos,et al.  New Additive Watermark Detectors Based On A Hierarchical Spatially Adaptive Image Model , 2008, IEEE Transactions on Information Forensics and Security.

[16]  Sailes K. Sengijpta.  Fundamentals of Statistical Signal Processing: Estimation Theory , 1995 .

[17]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[18]  Nikolas P. Galatsanos,et al.  A variational approach for Bayesian blind image deconvolution , 2004, IEEE Transactions on Signal Processing.

[19]  S. Stigler.  Laplace's 1774 Memoir on Inverse Probability , 1986 .

[20]  Aristidis Likas,et al.  Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting , 2007, IEEE Transactions on Neural Networks.

[21]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[22]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[23]  V. Šmídl,et al.  The Variational Bayes Method in Signal Processing , 2005 .

[24]  Nikolas P. Galatsanos,et al.  Bayesian Restoration Using a New Nonstationary Edge-Preserving Image Prior , 2006, IEEE Transactions on Image Processing.

[25]  S. Newcomb.  A Generalized Theory of the Combination of Observations so as to Obtain the Best Result , 1886 .

[26]  Aggelos K. Katsaggelos,et al.  Blind Deconvolution Using a Variational Approach to Parameter, Image, and Blur Estimation , 2006, IEEE Transactions on Image Processing.

[27]  Christopher M. Bishop,et al.  Variational Relevance Vector Machines , 2000, UAI.

[28]  Steve R. Waterhouse,et al.  Bayesian Methods for Mixtures of Experts , 1995, NIPS.

[29]  Nikolas P. Galatsanos,et al.  A spatially constrained mixture model for image segmentation , 2005, IEEE Transactions on Neural Networks.

[30]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[31]  G. Parisi,et al.  Statistical Field Theory , 1988 .

[32]  Nikolas P. Galatsanos,et al.  Variational Bayesian Blind Image Deconvolution with Student-T Priors , 2007, 2007 IEEE International Conference on Image Processing.

[33]  Stephen J. Roberts,et al.  Adaptive BCI based on variational Bayesian Kalman filtering: an empirical evaluation , 2004, IEEE Transactions on Biomedical Engineering.

[34]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[35]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[36]  A. Likas,et al.  Bayesian Image Restoration Based On Variatonal Inference and a Product of Student-t Priors , 2007, 2007 IEEE Workshop on Machine Learning for Signal Processing.

[37]  Nikolas P. Galatsanos,et al.  A Class-Adaptive Spatially Variant Mixture Model for Image Segmentation , 2007, IEEE Transactions on Image Processing.

[38]  Eric V. Slud.  Graphical Models: Methods for Data Analysis and Mining , 2003, Technometrics.

[39]  Naonori Ueda,et al.  Variational bayesian estimation and clustering for speech recognition , 2004, IEEE Transactions on Speech and Audio Processing.