Gaussian belief propagation solver for systems of linear equations

The canonical problem of solving a system of linear equations arises in numerous contexts in information theory, communication theory, and related fields. In this contribution, we develop a solution based upon Gaussian belief propagation (GaBP) that does not involve direct matrix inversion. The iterative nature of our approach allows for a distributed message-passing implementation of the solution algorithm. We also address some properties of the GaBP solver, including convergence, exactness, its max-product version and relation to classical solution methods. The application example of decorrelation in CDMA is used to demonstrate the faster convergence rate of the proposed solver in comparison to conventional linear-algebraic iterative solution methods.

[1]  Alex Grant,et al.  Iterative Implementations for Linear Multiuser Detectors , 1999 .

[2]  Dmitry M. Malioutov,et al.  Walk-Sum Interpretation and Analysis of Gaussian Belief Propagation , 2005, NIPS.

[3]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[4]  Dmitry M. Malioutov,et al.  Walk-Sums and Belief Propagation in Gaussian Graphical Models , 2006, J. Mach. Learn. Res..

[5]  Panganamala Ramana Kumar,et al.  Extended message passing algorithm for inference in loopy Gaussian graphical models , 2004, Ad Hoc Networks.

[6]  James K. Cavers,et al.  Multiuser channel estimation for detection of cochannel signals , 2001, IEEE Trans. Commun..

[7]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[8]  Lars K. Rasmussen,et al.  Linear interference cancellation in CDMA based on iterative techniques for linear equation systems , 2000, IEEE Trans. Commun..

[9]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[10]  Paul H. Siegel,et al.  Gaussian belief propagation based multiuser detection , 2008, 2008 IEEE International Symposium on Information Theory.

[11]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[12]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[13]  Rüdiger L. Urbanke,et al.  Modern Coding Theory , 2008 .

[14]  Roy D. Yates,et al.  CDMA multiuser detection: a nonlinear programming approach , 2002, IEEE Trans. Commun..