Approximate Contrastive Free Energies for Learning in Undirected Graphical Models

We present a novel class of learning algorithms for undirected graphical models, based on the contrastive free energy (CF). In particular we study the naive mean field, TAP and Bethe approximations to the contrastive free energy. The main advantage of CFlearning is the fact that it eliminates the need to infer equilibrium statistics for which mean field type approximations are particularly unsuitable. Instead, learning decreases the distance between the data distribution and the distribution with onestep reconstructions clamped on the visible nodes. We test the learning algorithm on the classification of digits.