Preconditioning in Expectation

We show that preconditioners constructed by random sampling can perform well without meeting the standard requirements of iterative methods. When applied to graph Laplacians, this leads to ultra-sparsifiers that in expectation behave as the nearly-optimal ones given by [Kolla-Makarychev-Saberi-Teng STOC`10]. Combining this with the recursive preconditioning framework by [Spielman-Teng STOC`04] and improved embedding algorithms, this leads to algorithms that solve symmetric diagonally dominant linear systems and electrical flow problems in expected time close to $m\log^{1/2}n$ .

[1]  Kunio Tanabe,et al.  Upper and lower bounds for the arithmetic-geometric-harmonic means of positive definite matrices , 1994 .

[2]  Ojas Parekh,et al.  Finding effective support-tree preconditioners , 2005, SPAA '05.

[3]  Amin Saberi,et al.  Subgraph sparsification and nearly optimal ultrasparsifiers , 2009, STOC '10.

[4]  Shang-Hua Teng,et al.  Lower-stretch spanning trees , 2004, STOC '05.

[5]  Léon Bottou,et al.  Stochastic Learning , 2003, Advanced Lectures on Machine Learning.

[6]  Ittai Abraham,et al.  Nearly Tight Low Stretch Spanning Trees , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[7]  Deanna Needell,et al.  Paved with Good Intentions: Analysis of a Randomized Block Kaczmarz Method , 2012, ArXiv.

[8]  Gary L. Miller,et al.  Approaching Optimality for Solving SDD Linear Systems , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[9]  Shang-Hua Teng,et al.  Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs , 2010, STOC '11.

[10]  Shang-Hua Teng,et al.  Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems , 2006, SIAM J. Matrix Anal. Appl..

[11]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[12]  Shang-Hua Teng,et al.  Spectral sparsification of graphs: theory and algorithms , 2013, CACM.

[13]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[14]  Shang-Hua Teng,et al.  The Laplacian Paradigm: Emerging Algorithms for Massive Graphs , 2010, TAMC.

[15]  D. Spielman.  Algorithms, Graph Theory, and Linear Equations in Laplacian Matrices , 2011 .

[16]  Noga Alon,et al.  A Graph-Theoretic Game and Its Application to the k-Server Problem , 1995, SIAM J. Comput..

[17]  Ittai Abraham,et al.  Using petal-decompositions to build a low stretch spanning tree , 2012, STOC '12.

[18]  Anastasios Zouzias,et al.  A Matrix Hyperbolic Cosine Algorithm and Applications , 2011, ICALP.

[19]  G. Miller,et al.  Combinatorial and algebraic tools for optimal multilevel algorithms , 2007 .

[20]  Yin Tat Lee,et al.  Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[21]  Zeyuan Allen Zhu,et al.  A simple, combinatorial algorithm for solving SDD systems in nearly-linear time , 2013, STOC '13.

[22]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[23]  Gary L. Miller,et al.  A Nearly-m log n Time Solver for SDD Linear Systems , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[24]  Nikhil Srivastava,et al.  Twice-ramanujan sparsifiers , 2008, STOC '09.

[25]  Daniel A. Spielman,et al.  A Note on Preconditioning by Low-Stretch Spanning Trees , 2009, ArXiv.

[26]  Shang-Hua Teng,et al.  Spectral Sparsification of Graphs , 2008, SIAM J. Comput..

[27]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[28]  David R. Karger,et al.  Approximating s-t minimum cuts in Õ(n2) time , 1996, STOC '96.

[29]  Richard Peng,et al.  Algorithm Design Using Spectral Graph Theory , 2013 .

[30]  O. Axelsson.  Iterative solution methods , 1995 .