Convergence of Gaussian Belief Propagation Under General Pairwise Factorization: Connecting Gaussian MRF with Pairwise Linear Gaussian Model

Gaussian belief propagation (BP) is a low-complexity and distributed method for computing the marginal distributions of a high-dimensional joint Gaussian distribution. However, Gaussian BP is only guaranteed to converge in singly connected graphs and may fail to converge in loopy graphs. Therefore, convergence analysis is a core topic in Gaussian BP. Existing conditions for verifying the convergence of Gaussian BP are all tailored for one particular pairwise factorization of the distribution in Gaussian Markov random field (MRF) and may not be valid for another pairwise factorization. On the other hand, convergence conditions of Gaussian BP in pairwise linear Gaussian model are developed independently from those in Gaussian MRF, making the convergence results highly scattered with diverse settings. In this paper, the convergence condition of Gaussian BP is investigated under a general pairwise factorization, which includes Gaussian MRF and pairwise linear Gaussian model as special cases. Upon this, existing convergence conditions in Gaussian MRF are extended to any pairwise factorization. Moreover, the newly established link between Gaussian MRF and pairwise linear Gaussian model reveals an easily verifiable sufficient convergence condition in pairwise linear Gaussian model, which provides a unified criterion for assessing the convergence of Gaussian BP in multiple applications. Numerical examples are presented to corroborate the theoretical results of this paper.

[1]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[2]  Dahlia Malkhi,et al.  Peer-to-Peer Rating , 2007, Seventh IEEE International Conference on Peer-to-Peer Computing (P2P 2007).

[3]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[4]  Dmitry M. Malioutov,et al.  Walk-Sums and Belief Propagation in Gaussian Graphical Models , 2006, J. Mach. Learn. Res..

[5]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[6]  Sekhar Tatikonda,et al.  Message-passing algorithms for quadratic minimization , 2012, J. Mach. Learn. Res..

[7]  Yik-Chung Wu,et al.  Distributed Clock Synchronization for Wireless Sensor Networks Using Belief Propagation , 2011, IEEE Transactions on Signal Processing.

[8]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[9]  Paul H. Siegel,et al.  Gaussian belief propagation based multiuser detection , 2008, 2008 IEEE International Symposium on Information Theory.

[10]  Yik-Chung Wu,et al.  Network-Wide Distributed Carrier Frequency Offsets Estimation and Compensation via Belief Propagation , 2013, IEEE Transactions on Signal Processing.

[11]  Yik-Chung Wu,et al.  Distributed Estimation of Variance in Gaussian Graphical Model via Belief Propagation: Accuracy Analysis and Improvement , 2015, IEEE Transactions on Signal Processing.

[12]  Paul H. Siegel,et al.  Gaussian belief propagation solver for systems of linear equations , 2008, 2008 IEEE International Symposium on Information Theory.

[13]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[14]  Benjamin Van Roy,et al.  Convergence of Min-Sum Message Passing for Quadratic Optimization , 2006, IEEE Transactions on Information Theory.

[15]  Benjamin Van Roy,et al.  Consensus Propagation , 2005, IEEE Transactions on Information Theory.

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  Yik-Chung Wu,et al.  On Convergence Conditions of Gaussian Belief Propagation , 2015, IEEE Transactions on Signal Processing.

[18]  Yik-Chung Wu,et al.  Convergence Analysis of the Variance in Gaussian Belief Propagation , 2014, IEEE Transactions on Signal Processing.

[19]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .