Analysis on the Boltzmann Machine with Random Input Drifts in Activation Function

The Boltzmann machine (BM) model is able to learn the probability distribution of input patterns. However, in analog realization, there are thermal noise and random offset voltages of amplifiers. Those realization issues affect the behaviour of the neurons’ activation function and they can be modelled as random input drifts. This paper analyzes the activation function and state distribution of BMs under the input random drift model. Since the state of a neuron is also determined by its activation function, the random input drifts may cause a BM to change the behaviour. We show that the effect of random input drifts is equivalent to raising temperature factor. Hence, from the Kullback–Leibler (KL) divergence perspective, we propose a compensation scheme to reduce the effect of random input drifts. In our derive of compensation scheme, we assume that the input drift follows the Gaussian distribution. Surprisedly, from our simulations, the proposed compensation scheme also works very well for other distributions.