The Boltzmann machine learning procedure has been successfully applied in deterministic networks of analog units that use a mean field approximation to efficiently simulate a truly stochastic system (Peterson and Anderson 1987). This type of deterministic Boltzmann machine (DBM) learns much faster than the equivalent stochastic Boltzmann machine (SBM), but since the learning procedure for DBM's is only based on an analogy with SBM's, there is no existing proof that it performs gradient descent in any function, and it has only been justified by simulations. By using the appropriate interpretation for the way in which a DBM represents the probability of an output vector given an input vector, it is shown that the DBM performs steepest descent in the same function as the original SBM, except at rare discontinuities. A very simple way of forcing the weights to become symmetrical is also described, and this makes the DBM more biologically plausible than back-propagation (Werbos 1974; Parker 1985; Rumelhart et al. 1986).
[1]
P. Werbos,et al.
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
,
1974
.
[2]
J J Hopfield,et al.
Neurons with graded response have collective computational properties like those of two-state neurons.
,
1984,
Proceedings of the National Academy of Sciences of the United States of America.
[3]
Geoffrey E. Hinton,et al.
Learning representations by back-propagating errors
,
1986,
Nature.
[4]
Geoffrey E. Hinton,et al.
Learning and relearning in Boltzmann machines
,
1986
.
[5]
Carsten Peterson,et al.
A Mean Field Theory Learning Algorithm for Neural Networks
,
1987,
Complex Syst..