Linear-Sized Sparsifiers via Near-Linear Time Discrepancy Theory

Discrepancy theory provides powerful tools for producing higher-quality objects which"beat the union bound"in fundamental settings throughout combinatorics and computer science. However, this quality has often come at the price of more expensive algorithms. We introduce a new framework for bridging this gap, by allowing for the efficient implementation of discrepancy-theoretic primitives. Our framework repeatedly solves regularized optimization problems to low accuracy to approximate the partial coloring method of [Rot17], and simplifies and generalizes recent work of [JSS23] on fast algorithms for Spencer's theorem. In particular, our framework only requires that the discrepancy body of interest has exponentially large Gaussian measure and is expressible as a sublevel set of a symmetric, convex function. We combine this framework with new tools for proving Gaussian measure lower bounds to give improved algorithms for a variety of sparsification and coloring problems. As a first application, we use our framework to obtain an $\widetilde{O}(m \cdot \epsilon^{-3.5})$ time algorithm for constructing an $\epsilon$-approximate spectral sparsifier of an $m$-edge graph, matching the sparsity of [BSS14] up to constant factors and improving upon the $\widetilde{O}(m \cdot \epsilon^{-6.5})$ runtime of [LeeS17]. We further give a state-of-the-art algorithm for constructing graph ultrasparsifiers and an almost-linear time algorithm for constructing linear-sized degree-preserving sparsifiers via discrepancy theory; in the latter case, such sparsifiers were not known to exist previously. We generalize these results to their analogs in sparsifying isotropic sums of positive semidefinite matrices. Finally, to demonstrate the versatility of our technique, we obtain a nearly-input-sparsity time constructive algorithm for Spencer's theorem (where we recover a recent result of [JSS23]).

[1]  A. Jambulapati,et al.  Revisiting Area Convexity: Faster Box-Simplex Games and Spectrahedral Generalizations , 2023, 2303.15627.

[2]  Adrian Vladu,et al.  Discrepancy Minimization via Regularization , 2022, SODA.

[3]  N. Bansal,et al.  Resolving Matrix Spencer Conjecture Up to Poly-logarithmic Rank , 2022, STOC.

[4]  Kasper Green Larsen Fast Discrepancy Minimization with Hereditary Guarantees , 2022, SODA.

[5]  A. Sah,et al.  Spencer's theorem in nearly input-sparsity time , 2022, SODA.

[6]  T. Rothvoss,et al.  Vector balancing in Lebesgue spaces , 2020, Random Struct. Algorithms.

[7]  Omri Weinstein,et al.  Discrepancy Minimization in Input-Sparsity Time , 2022, ArXiv.

[8]  S. Vempala,et al.  A Unified Approach to Discrepancy Minimization , 2022, APPROX/RANDOM.

[9]  Richard Peng,et al.  Sparsified block elimination for directed laplacians , 2021, STOC.

[10]  Daniel Dadush,et al.  A new framework for matrix discrepancy: partial coloring bounds via mirror descent , 2021, STOC.

[11]  Aaron Sidford,et al.  Semi-Streaming Bipartite Matching in Fewer Passes and Optimal Space , 2020, SODA.

[12]  Aaron Sidford,et al.  Ultrasparse Ultrasparsifiers and Faster Laplacian System Solvers , 2020, SODA.

[13]  Mehtaab Sawhney,et al.  Discrepancy minimization via a self-balancing walk , 2020, STOC.

[14]  Kevin Tian,et al.  Coordinate Methods for Matrix Games , 2020, 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS).

[15]  N. Bansal,et al.  Online vector balancing and geometric discrepancy , 2019, STOC.

[16]  Thomas Rothvoss,et al.  Linear Size Sparsifier and the Geometry of the Operator Norm Ball , 2019, SODA.

[17]  Mohit Singh,et al.  Sticky Brownian Rounding and its Applications to Constraint Satisfaction Problems , 2018, SODA.

[18]  Peng Zhang,et al.  Balancing covariates in randomized experiments using the Gram-Schmidt walk , 2019, ArXiv.

[19]  Kevin Tian,et al.  Variance Reduction for Matrix Games , 2019, NeurIPS.

[20]  Kevin Tian,et al.  A Rank-1 Sketch for Matrix Multiplicative Weights , 2019, COLT.

[21]  Di Wang,et al.  Expander Decomposition and Pruning: Faster, Stronger, and Simpler , 2018, SODA.

[22]  Sushant Sachdeva,et al.  Short Cycles via Low-Diameter Decompositions , 2018, SODA.

[23]  Luca Trevisan,et al.  Optimal Lower Bounds for Sketching Graph Cuts , 2017, SODA.

[24]  Eylon Yogev,et al.  Optimal Short Cycle Decomposition in Almost Linear Time , 2019, ICALP.

[25]  Aleksandar Nikolov,et al.  Balancing Vectors in Any Norm , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[26]  Richard Peng,et al.  Graph Sparsification, Spectral Sketches, and Faster Resistance Computation, via Short Cycle Decompositions , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[27]  Shachar Lovett,et al.  The Gram-Schmidt walk: a cure for the Banaszczyk blues , 2017, STOC.

[28]  Mohit Singh,et al.  Efficient algorithms for discrepancy minimization in convex sets , 2014, Random Struct. Algorithms.

[29]  Yin Tat Lee,et al.  An SDP-based algorithm for linear-sized spectral sparsification , 2017, STOC.

[30]  Yuanzhi Li,et al.  Follow the Compressed Leader: Faster Online Learning of Eigenvectors and Faster MMWU , 2017, ICML.

[31]  He Sun,et al.  Constructing Linear-Sized Spectral Sparsification in Almost-Linear Time , 2017 .

[32]  Richard Peng,et al.  Faster Algorithms for Computing the Stationary Distribution, Simulating Random Walks, and More , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[33]  Nikhil Bansal,et al.  An Algorithm for Komlós Conjecture Matching Banaszczyk's Bound , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[34]  Michael B. Cohen,et al.  Ramanujan Graphs in Polynomial Time , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[35]  Richard Peng,et al.  Approximate Undirected Maximum Flows in O(mpolylog(n)) Time , 2014, SODA.

[36]  Tengyu Ma,et al.  Online Learning of Eigenvectors , 2015, ICML.

[37]  Zeyuan Allen Zhu,et al.  Spectral Sparsification and Regret Minimization Beyond Matrix Multiplicative Updates , 2015, STOC.

[38]  Cameron Musco,et al.  Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition , 2015, NIPS.

[39]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[40]  Thomas Rothvoß,et al.  Constructive Discrepancy Minimization for Convex Sets , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[41]  Yin Tat Lee,et al.  An Almost-Linear-Time Algorithm for Approximate Max Flow in Undirected Graphs, and its Multicommodity Generalizations , 2013, SODA.

[42]  Mohit Singh,et al.  Discrepancy Without Partial Colorings , 2014, APPROX-RANDOM.

[43]  Arkadi Nemirovski,et al.  A Randomized Mirror-Prox Method for Solving Structured Large-Scale Matrix Saddle-Point Problems , 2011, SIAM J. Optim..

[44]  Ittai Abraham,et al.  Using petal-decompositions to build a low stretch spanning tree , 2012, STOC '12.

[45]  Shachar Lovett,et al.  Constructive Discrepancy Minimization by Walking on the Edges , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[46]  Gary L. Miller,et al.  A Nearly-m log n Time Solver for SDD Linear Systems , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[47]  Nikhil Srivastava,et al.  Graph sparsification by effective resistances , 2008, SIAM J. Comput..

[48]  David P. Woodruff,et al.  Sublinear Optimization for Machine Learning , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[49]  Nikhil Bansal,et al.  Constructive Algorithms for Discrepancy Minimization , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[50]  Amin Saberi,et al.  Subgraph sparsification and nearly optimal ultrasparsifiers , 2009, STOC '10.

[51]  J. Matousek,et al.  Geometric Discrepancy: An Illustrated Guide , 2009 .

[52]  Daniel A. Spielman,et al.  A Note on Preconditioning by Low-Stretch Spanning Trees , 2009, ArXiv.

[53]  Yurii Nesterov,et al.  Smoothing Technique and its Applications in Semidefinite Optimization , 2004, Math. Program..

[54]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[55]  Santosh S. Vempala,et al.  Efficient algorithms for online decision problems , 2005, J. Comput. Syst. Sci..

[56]  Bernard Chazelle,et al.  The discrepancy method - randomness and complexity , 2000 .

[57]  B. Bollobás THE VOLUME OF CONVEX BODIES AND BANACH SPACE GEOMETRY (Cambridge Tracts in Mathematics 94) , 1991 .

[58]  G. Pisier The volume of convex bodies and Banach space geometry , 1989 .

[59]  J. Spencer Six standard deviations suffice , 1985 .

[60]  Robert E. Tarjan,et al.  A data structure for dynamic trees , 1981, STOC '81.