UTML TR 2011 – 002 Data Normalization in the Learning of Restricted Boltzmann Machines

In practice, training Restricted Boltzmann Machines with Contrastive Divergence and other approximate maximum likelihood methods works well on data with black backgrounds. However, when using inverted images for training, learning is typically much worse. In this paper, we propose a very simple yet very effective solution to this problem. The new algorithm requires the addition of only three(!) lines of code to existing RBM learning algorithms. Data Normalization in the Learning of Restricted Boltzmann Machines Yichuan Tang, and Ilya Sutskever Department of Computer Science, University of Toronto