Bayes Blocks: An Implementation of the Variational Bayesian Building Blocks Framework

A software library for constructing and learning probabilistic models is presented. The library offers a set of building blocks from which a large variety of static and dynamic models can be built. These include hierarchical models for variances of other variables and many nonlinear models. The underlying variational Bayesian machinery, providing for fast and robust estimation but being mathematically rather involved, is almost completely hidden from the user thus making it very easy to use the library. The building blocks include Gaussian, rectified Gaussian and mixture-of-Gaussians variables and computational nodes which can be combined rather freely.

[1]  Antti Honkela,et al.  Variational learning and bits-back coding: an information-theoretic view to Bayesian learning , 2004, IEEE Transactions on Neural Networks.

[2]  Lucas C. Parra,et al.  Higher-Order Statistical Properties Arising from the Non-Stationarity of Natural Signals , 2000, NIPS.

[3]  Juha Karhunen,et al.  Building Blocks for Variational Bayesian Learning of Latent Variable Models , 2007, J. Mach. Learn. Res..

[4]  J. Karhunen,et al.  Nonlinear Independent Factor Analysis by Hierarchical Models , 2003 .

[5]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[6]  J. Karhunen,et al.  Building Blocks for Hierarchical Latent Variable Models , 2001 .

[7]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[8]  D. Ruderman,et al.  Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[9]  David Barber,et al.  Ensemble Learning for Multi-Layer Networks , 1997, NIPS.

[10]  Juha Karhunen,et al.  Hierarchical models of variance sources , 2004, Signal Process..

[11]  Juha Karhunen,et al.  Accelerating Cyclic Update Algorithms for Parameter Estimation by Pattern Searches , 2003, Neural Processing Letters.

[12]  Charles M. Bishop,et al.  Variational Message Passing , 2005, J. Mach. Learn. Res..

[13]  Juha Karhunen,et al.  An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models , 2002, Neural Computation.

[14]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[15]  David J. Spiegelhalter,et al.  VIBES: A Variational Inference Engine for Bayesian Networks , 2002, NIPS.

[16]  Astronomy,et al.  A data-driven Bayesian approach for finding young stellar populations in early-type galaxies from their ultraviolet-optical spectra , 2005, astro-ph/0511503.

[17]  Robert Cowell,et al.  Introduction to Inference for Bayesian Networks , 1998, Learning in Graphical Models.

[18]  Markus Harva,et al.  Hierarchical Variance Models of Image Sequences , 2004 .

[19]  A. Kabán,et al.  A variational Bayesian method for rectified factor analysis , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[20]  Geoffrey E. Hinton,et al.  Variational Learning for Switching State-Space Models , 2000, Neural Computation.

[21]  T. Raiko,et al.  Partially observed values , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[22]  Juha Karhunen,et al.  Missing Values in Hierarchical Nonlinear Factor Analysis , 2003 .

[23]  Antti Honkela,et al.  Bayes Blocks Software Library , 2003 .