论文信息 - A New Learning Algorithm for Mean Field Boltzmann Machines

A New Learning Algorithm for Mean Field Boltzmann Machines

We present a new learning algorithm for Mean Field Boltzmann Machines based on the contrastive divergence optimization criterion. In addition to minimizing the divergence between the data distribution and the equilibrium distribution, we maximize the divergence between one-step reconstructions of the data and the equilibrium distribution. This eliminates the need to estimate equilibrium statistics, so we do not need to approximate the multimodal probability distribution of the free network with the unimodal mean field distribution. We test the learning algorithm on the classification of digits.

Geoffrey E. Hinton | Max Welling | M. Welling

[1] J J Hopfield,et al. Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[2] Geoffrey E. Hinton,et al. A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[3] Geoffrey E. Hinton,et al. Learning and relearning in Boltzmann machines , 1986 .

[4] Carsten Peterson,et al. A Mean Field Theory Learning Algorithm for Neural Networks , 1987, Complex Syst..

[5] Geoffrey E. Hinton. Deterministic Boltzmann Learning Performs Steepest Descent in Weight-Space , 1989, Neural Computation.

[6] Javier R. Movellan,et al. Contrastive Hebbian Learning in the Continuous Hopfield Model , 1991 .

[7] C. Galland. The limitations of deterministic Boltzmann machine learning , 1993 .

[8] Hilbert J. Kappen,et al. Efficient Learning in Boltzmann Machines Using Linear Response Theory , 1998, Neural Computation.

[9] Yee Whye Teh,et al. A New View of ICA , 2001 .

[10] Marco Zorzi,et al. Associative Arithmetic with Boltzmann Machines: The Role of Number Representations , 2002, ICANN.

[11] Geoffrey E. Hinton. Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.