An analysis of Gaussian-binary restricted Boltzmann machines for natural images

A Gaussian-binary restricted Boltzmann machine is a widely used energy-based model for continuous data distributions, although many authors reported difficulties in training on natural images. To clarify the model’s capabilities and limitations we derive a rewritten formula of the probability density function as a linear superposition of Gaussians. Based on this formula we show how Gaussian-binary RBMs learn natural image statistics. However the probability density function of the model is not a good representation of the data distribution.