论文信息 - The Nonnegative Boltzmann Machine

The Nonnegative Boltzmann Machine

The nonnegative Boltzmann machine (NNBM) is a recurrent neural network model that can describe multimodal nonnegative data. Application of maximum likelihood estimation to this model gives a learning rule that is analogous to the binary Boltzmann machine. We examine the utility of the mean field approximation for the NNBM, and describe how Monte Carlo sampling techniques can be used to learn its parameters. Reflective slice sampling is particularly well-suited for this distribution, and can efficiently be implemented to sample the distribution. We illustrate learning of the NNBM on a transiationally invariant distribution, as well as on a generative model for images of human faces.

Daniel D. Lee | David J. C. MacKay | Oliver B. Downs | Daniel D. Lee | D. MacKay

[1] Radford M. Neal. Markov Chain Monte Carlo Methods Based on `Slicing' the Density Function , 1997 .

[2] H. Sebastian Seung,et al. Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[3] Radford M. Neal,et al. Suppressing Random Walks in Markov Chain Monte Carlo Using Ordered Overrelaxation , 1995, Learning in Graphical Models.

[4] David J. C. Mackay,et al. Introduction to Monte Carlo Methods , 1998, Learning in Graphical Models.

[5] H. Sompolinsky,et al. Theory of orientation tuning in visual cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[6] Hilbert J. Kappen,et al. Mean field approach to learning in Boltzmann Machine , 1997, Pattern Recognit. Lett..

[7] H. Sebastian Seung,et al. The Rectified Gaussian Distribution , 1997, NIPS.

[8] C. Galland. The limitations of deterministic Boltzmann machine learning , 1993 .

[9] Michael I. Jordan. Learning in Graphical Models , 1999, NATO ASI Series.

[10] Geoffrey E. Hinton,et al. A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[11] Hilbert J. Kappen,et al. Efficient Learning in Boltzmann Machines Using Linear Response Theory , 1998, Neural Computation.