An efficient parallel solver for SDD linear systems

We present the first parallel algorithm for solving systems of linear equations in symmetric, diagonally dominant (SDD) matrices that runs in polylogarithmic time and nearly-linear work. The heart of our algorithm is a construction of a sparse approximate inverse chain for the input matrix: a sequence of sparse matrices whose product approximates its inverse. Whereas other fast algorithms for solving systems of equations in SDD matrices exploit low-stretch spanning trees, our algorithm only requires spectral graph sparsifiers.

[1]  Mark Rudelson,et al.  Sampling from large matrices: An approach through geometric functional analysis , 2005, JACM.

[2]  Zeyuan Allen Zhu,et al.  A simple, combinatorial algorithm for solving SDD systems in nearly-linear time , 2013, STOC '13.

[3]  William L. Briggs,et al.  A multigrid tutorial, Second Edition , 2000 .

[4]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[5]  Aleksander Madry,et al.  Navigating Central Path with Electrical Flows: From Flows to Matchings, and Back , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[6]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[7]  Gary L. Miller,et al.  Approaching Optimality for Solving SDD Linear Systems , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[8]  Jonathan A. Kelner,et al.  Spectral Sparsification in the Semi-streaming Setting , 2012, Theory of Computing Systems.

[9]  M. Rudelson.  Random Vectors in the Isotropic Position , 1996, math/9608208.

[10]  Bruce Hendrickson,et al.  Support Theory for Preconditioning , 2003, SIAM J. Matrix Anal. Appl..

[11]  Shang-Hua Teng,et al.  A Local Clustering Algorithm for Massive Graphs and Its Application to Nearly Linear Time Graph Partitioning , 2008, SIAM J. Comput..

[12]  Yin Tat Lee,et al.  Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[13]  Nikhil Srivastava,et al.  Graph sparsification by effective resistances , 2008, SIAM J. Comput..

[14]  Shang-Hua Teng,et al.  Spectral Sparsification of Graphs , 2008, SIAM J. Comput..

[15]  B. Schölkopf,et al.  A Regularization Framework for Learning from Graph Data , 2004, ICML 2004.

[16]  Nikhil Srivastava,et al.  Twice-ramanujan sparsifiers , 2008, STOC '09.

[17]  Sivan Toledo,et al.  Support-Graph Preconditioners , 2005, SIAM J. Matrix Anal. Appl..

[18]  Anil Joshi.  Topics in optimization and sparse linear systems , 1997 .

[19]  Gary L. Miller,et al.  Faster approximate multicommodity flow using quadratically coupled flows , 2012, STOC '12.

[20]  Daniel A. Spielman,et al.  Faster approximate lossy generalized flow via interior point algorithms , 2008, STOC.

[21]  Bruce Hendrickson,et al.  Solving Elliptic Finite Element Systems in Near-Linear Time with Support Preconditioners , 2004, SIAM J. Numer. Anal..

[22]  J. Reif.  Efficient approximate solution of sparse linear systems , 1998 .

[23]  Nisheeth K. Vishnoi,et al.  Towards an SDP-based approach to spectral methods: a nearly-linear-time algorithm for graph partitioning and decomposition , 2010, SODA '11.

[24]  Aleksander Madry,et al.  Faster Generation of Random Spanning Trees , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[25]  Shang-Hua Teng,et al.  Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems , 2006, SIAM J. Matrix Anal. Appl..

[26]  Gary L. Miller,et al.  Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing , 2009, Comput. Vis. Image Underst..

[27]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[28]  Gary L. Miller,et al.  A Nearly-m log n Time Solver for SDD Linear Systems , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[29]  Nisheeth K. Vishnoi,et al.  Approximating the exponential, the lanczos method and an Õ(m)-time spectral algorithm for balanced separator , 2011, STOC '12.

[30]  A. George,et al.  Graph theory and sparse matrix computation , 1993 .

[31]  O. Axelsson.  Iterative solution methods , 1995 .

[32]  S. McCormick,et al.  A multigrid tutorial (2nd ed.) , 2000 .

[33]  Shang-Hua Teng,et al.  Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs , 2010, STOC '11.

[34]  Richard Peng,et al.  Improved Spectral Sparsification and Numerical Algorithms for SDD Matrices , 2012, STACS.

[35]  Gary L. Miller,et al.  A linear work, O(n1/6) time, parallel algorithm for solving planar Laplacians , 2007, SODA '07.