Variations on the Boltzmann Machine Learning Algorithm

Abstract : Boltzmann Machines learn to model the structure of an environment by modifying internal weights. The algorithm used for changing a weight depends on collecting statistics about the behavior of the two units that the weight connects. The success and speed of the algorithm depends on the accuracy of the statistics, the size of the weight changes, and the way in which the accuracy of the machine's model varies as the weights are changed. This paper presents theoretical analysis and empirical results that can be used to select more effective parameters for the learning algorithm. The Boltzmann Machine learns to make its model of its environment correspond to the actual environment in which it is placed. An environment is a probability distribution of patterns over a subset of the units called the visible units. The environment can be specified explicitly as a list of ordered pairs, giving a pattern, V a' over the visible units, and a probability for the occurrence of the pattern. The machine's model is just the probability distribution it would produce over the visible units if it were allowed to run freely without any environmental input. This probability distribution is not stored directly. Instead, it is specified implicitly by the magnitude of the weights in the machine. For a machine architecture with v visible units, there are 2(v) possible patterns. A machine whose weights were all zero would implicitly specify an environment where each of these patterns had probability 2-v, since the energy of all states would be the same.