The Convergence of Contrastive Divergences

The Convergence of Contrastive Divergences Alan Yuille Department of Statistics University of California at Los Angeles Los Angeles, CA 90095 yuille@stat.ucla.edu Abstract This paper analyses the Contrastive Divergence algorithm for learning statistical parameters. We relate the algorithm to the stochastic approxi- mation literature. This enables us to specify conditions under which the algorithm is guaranteed to converge to the optimal solution (with proba- bility 1). This includes necessary and sufficient conditions for the solu- tion to be unbiased. 1 Introduction Many learning problems can be reduced to statistical inference of parameters. But inference algorithms for this task tend to be very slow. Recently Hinton proposed a new algorithm called contrastive divergences (CD) [1]. Computer simulations show that this algorithm tends to converge, and to converge rapidly, although not always to the correct solution [2]. Theoretical analysis shows that CD can fail but does not give conditions which guarantee convergence [3,4]. This paper relates CD to the stochastic approximation literature [5,6] and hence derives elementary conditions which ensure convergence (with probability 1). We conjecture that far stronger results can be obtained by applying more advanced techniques such as those described by Younes [7]. We also give necessary and sufficient conditions for the solution of CD to be unbiased. Section (2) describes CD and shows that it is closely related to a class of stochastic ap- proximation algorithms for which convergence results exist. In section (3) we state and give a proof of a simple convergence theorem for stochastic approximation algorithms. Section (4) applies the theorem to give sufficient conditions for convergence of CD. 2 Contrastive Divergence and its Relations The task of statistical inference is to estimate the model parameters ω ∗ which minimize the Kullback-Leibler divergence D(P 0 (x)||P (x|ω)) between the empirical distribution func-