Efficient Structured Matrix Recovery and Nearly-Linear Time Algorithms for Solving Inverse Symmetric M-Matrices

In this paper we show how to recover a spectral approximations to broad classes of structured matrices using only a polylogarithmic number of adaptive linear measurements to either the matrix or its inverse. Leveraging this result we obtain faster algorithms for variety of linear algebraic problems. Key results include: $\bullet$ A nearly linear time algorithm for solving the inverse of symmetric $M$-matrices, a strict superset of Laplacians and SDD matrices. $\bullet$ An $\tilde{O}(n^2)$ time algorithm for solving $n \times n$ linear systems that are constant spectral approximations of Laplacians or more generally, SDD matrices. $\bullet$ An $\tilde{O}(n^2)$ algorithm to recover a spectral approximation of a $n$-vertex graph using only $\tilde{O}(1)$ matrix-vector multiplies with its Laplacian matrix. The previous best results for each problem either used a trivial number of queries to exactly recover the matrix or a trivial $O(n^\omega)$ running time, where $\omega$ is the matrix multiplication constant. We achieve these results by generalizing recent semidefinite programming based linear sized sparsifier results of Lee and Sun (2017) and providing iterative methods inspired by the semistreaming sparsification results of Kapralov, Lee, Musco, Musco and Sidford (2014) and input sparsity time linear system solving results of Li, Miller, and Peng (2013). We hope that by initiating study of these natural problems, expanding the robustness and scope of recent nearly linear time linear system solving research, and providing general matrix recovery machinery this work may serve as a stepping stone for faster algorithms.

[1]  Gary L. Miller,et al.  A Nearly-m log n Time Solver for SDD Linear Systems , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[2]  Neal E. Young Nearly Linear-Time Approximation Schemes for Mixed Packing/Covering and Facility-Location Linear Programs , 2014, ArXiv.

[3]  Huy L. Nguyen,et al.  OSNAP: Faster Numerical Linear Algebra Algorithms via Sparser Subspace Embeddings , 2012, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[4]  Yin Tat Lee,et al.  Using Optimization to Obtain a Width-Independent, Parallel, Simpler, and Faster Positive SDP Solver , 2015, SODA.

[5]  Yin Tat Lee,et al.  Path Finding Methods for Linear Programming: Solving Linear Programs in Õ(vrank) Iterations and Faster Algorithms for Maximum Flow , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[6]  Shaun M. Fallat,et al.  Total positivity in Markov structures , 2015, 1510.01290.

[7]  Bruce Hendrickson,et al.  Solving Elliptic Finite Element Systems in Near-Linear Time with Support Preconditioners , 2004, SIAM J. Numer. Anal..

[8]  Yin Tat Lee,et al.  An Almost-Linear-Time Algorithm for Approximate Max Flow in Undirected Graphs, and its Multicommodity Generalizations , 2013, SODA.

[9]  Matthias Hein,et al.  Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov Random fields , 2014, 1404.6640.

[10]  Amin Saberi,et al.  Perron-Frobenius Theory in Nearly Linear Time: Positive Eigenvectors, M-matrices, Graph Kernels, and Other Applications , 2018, SODA.

[11]  Nisheeth K. Vishnoi,et al.  Towards an SDP-based approach to spectral methods: a nearly-linear-time algorithm for graph partitioning and decomposition , 2010, SODA '11.

[12]  John Peebles,et al.  Sampling random spanning trees faster than matrix multiplication , 2016, STOC.

[13]  Richard Peng,et al.  Almost-linear-time algorithms for Markov chains and new spectral primitives for directed graphs , 2016, STOC.

[14]  Piotr Sankowski,et al.  Negative-Weight Shortest Paths and Unit Capacity Minimum Cost Flow in Õ (m10/7 log W) Time (Extended Abstract) , 2016, SODA.

[15]  Aleksander Madry,et al.  Matrix Scaling and Balancing via Box Constrained Newton's Method and Interior Point Methods , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[16]  Gary L. Miller,et al.  Approaching Optimality for Solving SDD Linear Systems , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[17]  S. Karlin,et al.  M-Matrices as covariance matrices of multinormal distributions☆ , 1983 .

[18]  Shang-Hua Teng,et al.  Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs , 2010, STOC '11.

[19]  Richard Peng,et al.  Faster Algorithms for Computing the Stationary Distribution, Simulating Random Walks, and More , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[20]  Richard Peng,et al.  An efficient parallel solver for SDD linear systems , 2013, STOC.

[21]  Richard Peng,et al.  Sparsified Cholesky and multigrid solvers for connection laplacians , 2015, STOC.

[22]  Charalampos E. Tsourakakis,et al.  Learning Networks from Random Walk-Based Node Similarities , 2018, ArXiv.

[23]  Gary L. Miller,et al.  Iterative Row Sampling , 2012, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[24]  He Sun Constructing Linear-Sized Spectral Sparsification in Almost-Linear Time , 2017 .

[25]  Rahul Jain,et al.  A parallel approximation algorithm for mixed packing and covering semidefinite programs , 2012, ArXiv.

[26]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[27]  Shang-Hua Teng,et al.  The Laplacian Paradigm: Emerging Algorithms for Massive Graphs , 2010, TAMC.

[28]  Daniel A. Spielman,et al.  Algorithms for Lipschitz Learning on Graphs , 2015, COLT.

[29]  Yu Cheng,et al.  Scalable Parallel Factorizations of SDD Matrices and Efficient Sampling for Gaussian Graphical Models , 2014, ArXiv.

[30]  Richard Peng,et al.  Faster and simpler width-independent parallel algorithms for positive semidefinite programming , 2012, SPAA '12.

[31]  Jakub W. Pachocki,et al.  Solving SDD linear systems in nearly mlog1/2n time , 2014, STOC.

[32]  Yin Tat Lee,et al.  An SDP-based algorithm for linear-sized spectral sparsification , 2017, STOC.

[33]  David P. Woodruff,et al.  On Sketching Quadratic Forms , 2015, ITCS.

[34]  Aleksander Madry,et al.  Faster Generation of Random Spanning Trees , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[35]  Satish Rao,et al.  A new approach to computing maximum flows using electrical flows , 2013, STOC '13.

[36]  Nikhil Srivastava,et al.  Graph Sparsification by Effective Resistances , 2011, SIAM J. Comput..

[37]  Richard Peng,et al.  Uniform Sampling for Matrix Approximation , 2014, ITCS.

[38]  Sushant Sachdeva,et al.  Approximate Gaussian Elimination for Laplacians - Fast, Sparse, and Simple , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[39]  Yu Cheng,et al.  Non-Convex Matrix Completion Against a Semi-Random Adversary , 2018, COLT.

[40]  Nisheeth K. Vishnoi,et al.  Approximating the exponential, the lanczos method and an Õ(m)-time spectral algorithm for balanced separator , 2011, STOC '12.

[41]  Virginia Vassilevska Williams,et al.  Multiplying matrices faster than coppersmith-winograd , 2012, STOC '12.

[42]  Yin Tat Lee,et al.  Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[43]  Aaron Schild,et al.  An almost-linear time algorithm for uniform random spanning tree generation , 2017, STOC.

[44]  Daniel A. Spielman,et al.  Faster approximate lossy generalized flow via interior point algorithms , 2008, STOC.

[45]  Rahul Jain,et al.  A Parallel Approximation Algorithm for Positive Semidefinite Programming , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[46]  Zeyuan Allen Zhu,et al.  Spectral Sparsification and Regret Minimization Beyond Matrix Multiplicative Updates , 2015, STOC.

[47]  Aleksander Madry,et al.  Computing Maximum Flow with Augmenting Electrical Flows , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[48]  Zeyuan Allen Zhu,et al.  A simple, combinatorial algorithm for solving SDD systems in nearly-linear time , 2013, STOC '13.

[49]  Yin Tat Lee,et al.  Single Pass Spectral Sparsification in Dynamic Streams , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.