A new approach to Laplacian solvers and flow problems

This paper investigates the behavior of the Min-Sum message passing scheme to solve systems of linear equations in the Laplacian matrices of graphs and to compute electric flows. Voltage and flow problems involve the minimization of quadratic functions and are fundamental primitives that arise in several domains. Algorithms that have been proposed are typically centralized and involve multiple graph-theoretic constructions or sampling mechanisms that make them difficult to implement and analyze. On the other hand, message passing routines are distributed, simple, and easy to implement. In this paper we establish a framework to analyze Min-Sum to solve voltage and flow problems. We characterize the error committed by the algorithm on general weighted graphs in terms of hitting times of random walks defined on the computation trees that support the operations of the algorithms with time. For $d$-regular graphs with equal weights, we show that the convergence of the algorithms is controlled by the total variation distance between the distributions of non-backtracking random walks defined on the original graph that start from neighboring nodes. The framework that we introduce extends the analysis of Min-Sum to settings where the contraction arguments previously considered in the literature (based on the assumption of walk summability or scaled diagonal dominance) can not be used, possibly in the presence of constraints.

[1]  Richard Peng,et al.  An efficient parallel solver for SDD linear systems , 2013, STOC.

[2]  Devavrat Shah,et al.  Belief propagation for min-cost network flow: convergence & correctness , 2010, SODA '10.

[3]  Benjamin Van Roy,et al.  Convergence of Min-Sum Message-Passing for Convex Optimization , 2010, IEEE Transactions on Information Theory.

[4]  Andrea Montanari,et al.  Graphical Models Concepts in Compressed Sensing , 2010, Compressed Sensing.

[5]  Gary L. Miller,et al.  A Nearly-m log n Time Solver for SDD Linear Systems , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[6]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[7]  Emmanuel Abbe,et al.  Community Detection and Stochastic Block Models , 2017, Found. Trends Commun. Inf. Theory.

[8]  Dmitry M. Malioutov,et al.  Walk-Sums and Belief Propagation in Gaussian Graphical Models , 2006, J. Mach. Learn. Res..

[9]  Riccardo Zecchina,et al.  Survey propagation: An algorithm for satisfiability , 2002, Random Struct. Algorithms.

[10]  Benjamin Van Roy,et al.  Consensus Propagation , 2005, IEEE Transactions on Information Theory.

[11]  Benjamin Van Roy,et al.  An analysis of belief propagation on the turbo decoding graph with Gaussian densities , 2001, IEEE Trans. Inf. Theory.

[12]  Jakub W. Pachocki,et al.  Solving SDD linear systems in nearly mlog1/2n time , 2014, STOC.

[13]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[14]  C MoallemiCiamac,et al.  Convergence of min-sum message-passing for convex optimization , 2010 .

[15]  D. Spielman.  Algorithms, Graph Theory, and Linear Equations in Laplacian Matrices , 2011 .

[16]  Shang-Hua Teng,et al.  Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems , 2006, SIAM J. Matrix Anal. Appl..

[17]  V. Climenhaga.  Markov chains and mixing times , 2013 .

[18]  Benjamin Van Roy,et al.  Convergence of Min-Sum Message Passing for Quadratic Optimization , 2006, IEEE Transactions on Information Theory.

[19]  Zeyuan Allen Zhu,et al.  A simple, combinatorial algorithm for solving SDD systems in nearly-linear time , 2013, STOC '13.

[20]  Sekhar Tatikonda,et al.  Loopy Belief Propogation and Gibbs Measures , 2002, UAI.

[21]  Sushant Sachdeva,et al.  Approximate Gaussian Elimination for Laplacians - Fast, Sparse, and Simple , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[22]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[23]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[24]  Peter G. Doyle,et al.  Random Walks and Electric Networks: REFERENCES , 1987 .

[25]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[26]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[27]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[28]  Nicolas Macris,et al.  Optimal errors and phase transitions in high-dimensional generalized linear models , 2017, Proceedings of the National Academy of Sciences.