An Edge Deletion Semantics for Belief Propagation

Iterative belief propagation is an influential method for approximate inference in probabilistic graphical models, perhaps the most influential method of the last decade. Given its wide-spread applicability in various domains, there has been a great interest in developing semantics for this method to both characterize and control the quality of its approximations. We present in this paper a new semantics for belief propagation, formalizing it as a method of exact inference on a simplified model that has been obtained by deleting edges from the original. When we delete an edge, however, we lose a model dependency, which we can compensate for by adding auxiliary parameters to the model. We show that the iterations of belief propagation are searching for such auxiliary parameters in a model which results from deleting every network edge. This semantics leads to a number of questions: Can we delete fewer than every model edge? Further, which edges should we delete and which should we let remain? The answers to these questions lead to a generalization of belief propagation based on edge deletion, which we present here, allowing one to trade approximation quality with computational resources. This edge deletion perspective sheds new light on belief propagation approximations, and further enables an effective procedure for finding improved approximations through a simple process of edge recovery.