In an important recent paper, Yedidia, Freeman, and Weiss [7] showed that there is a close connection between the belief propagation algorithm for probabilistic inference and the Bethe-Kikuchi approximation to the variational free energy in statistical physics. In this paper, we will recast the YFW results in the context of the “generalized distributive law” [1] formulation of belief propagation. Our main result is that if the GDL is applied to junction graph, the fixed points of the algorithm are in one-to-one correspondence with the stationary points of a certain Bethe-Kikuchi free energy. If the junction graph has no cycles, the BK free energy is convex and has a unique stationary point, which is a global minimum. On the other hand, if the junction graph has cycles, the main result at least shows that the GDL is trying to do something sensible.
[1]
C. Thompson.
The Statistical Mechanics of Phase Transitions
,
1978
.
[2]
Judea Pearl,et al.
Probabilistic reasoning in intelligent systems
,
1988
.
[3]
Thomas M. Cover,et al.
Elements of Information Theory
,
2005
.
[4]
Robert J. McEliece,et al.
The generalized distributive law
,
2000,
IEEE Trans. Inf. Theory.
[5]
J. Yedidia.
An Idiosyncratic Journey Beyond Mean Field Theory
,
2000
.
[6]
W. Freeman,et al.
Generalized Belief Propagation
,
2000,
NIPS.
[7]
M. Opper,et al.
An Idiosyncratic Journey Beyond Mean Field Theory
,
2001
.
[8]
W. Freeman,et al.
Bethe free energy, Kikuchi approximations, and belief propagation algorithms
,
2001
.