论文信息 - Stable Fixed Points of Loopy Belief Propagation Are Minima of the Bethe Free Energy

Stable Fixed Points of Loopy Belief Propagation Are Minima of the Bethe Free Energy

We extend recent work on the connection between loopy belief propagation and the Bethe free energy. Constrained minimization of the Bethe free energy can be turned into an unconstrained saddle-point problem. Both converging double-loop algorithms and standard loopy belief propagation can be interpreted as attempts to solve this saddle-point problem. Stability analysis then leads us to conclude that stable fixed points of loopy belief propagation must be (local) minima of the Bethe free energy. Perhaps surprisingly, the converse need not be the case: minima can be unstable fixed points. We illustrate this with an example and discuss implications.

T. Heskes

[1] Judea Pearl,et al. Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[2] Jeffrey C. Lagarias,et al. Minimax and Hamiltonian Dynamics of Excitatory-Inhibitory Networks , 1997, NIPS.

[3] Michael I. Jordan,et al. Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[4] W. Freeman,et al. Generalized Belief Propagation , 2000, NIPS.

[5] Brendan J. Frey,et al. Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[6] Yee Whye Teh,et al. The Unified Propagation and Scaling Algorithm , 2001, NIPS.

[7] Thomas P. Minka,et al. The EP energy function and minimization schemes , 2001 .

[8] T. Heskes,et al. Expectation propagation for approximate inference in dynamic bayesian networks , 2002, UAI 2002.

[9] Alan L. Yuille,et al. CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation , 2002, Neural Computation.