Computing Lewis Weights to High Precision

We present an algorithm for computing approximate `p Lewis weights to high precision. Given a full-rank A ∈ Rm×n with m ≥ n and a scalar p > 2, our algorithm computes approximate `p Lewis weights of A in Õp(log(1/ )) iterations; the cost of each iteration is linear in the input size plus the cost of computing the leverage scores of DA for diagonal D ∈ Rm×m. Prior to our work, such a computational complexity was known only for p ∈ (0, 4) [CP15], and combined with this result, our work yields the first polylogarithmic-depth polynomial-work algorithm for the problem of computing `p Lewis weights to high precision for all constant p > 0. An important consequence of this result is also the first polylogarithmic-depth polynomial-work algorithm for computing a nearly optimal self-concordant barrier for a polytope. mfazel@uw.edu. University of Washington. yintat@uw.edu. University of Washington. pswati@uw.edu. University of Washington. sidford@stanford.edu. Stanford University ar X iv :2 11 0. 15 56 3v 1 [ cs .D S] 2 9 O ct 2 02 1

[1]  F. John.  Extremum Problems with Inequalities as Subsidiary Conditions , 2014 .

[2]  David P. Woodruff,et al.  Near Optimal Linear Algebra in the Online and Sliding Window Models , 2020, 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS).

[3]  Maryam Fazel,et al.  Fair Packing and Covering on a Relative Scale , 2020, SIAM J. Optim..

[4]  Yuanzhi Li,et al.  Near-Optimal Design of Experiments via Regret Minimization , 2017, ICML.

[5]  Hariharan Narayanan,et al.  Random walks on polytopes and an affine interior point method for linear programming , 2009, STOC '09.

[6]  Peng Sun,et al.  Linear convergence of a modified Frank–Wolfe algorithm for computing minimum-volume enclosing ellipsoids , 2008, Optim. Methods Softw..

[7]  Peng Sun,et al.  Computation of Minimum Volume Covering Ellipsoids , 2002, Oper. Res..

[8]  Michael Jackson,et al.  Optimal Design of Experiments , 1994 .

[9]  Sébastien Bubeck,et al.  The entropic barrier: a simple and optimal universal self-concordant barrier , 2014, COLT.

[10]  Elad Hazan,et al.  Competing in the Dark: An Efficient Algorithm for Bandit Linear Optimization , 2008, COLT.

[11]  D. R. Lewis.  Finite dimensional subspaces of $L_{p}$ , 1978 .

[12]  Michael J. Todd,et al.  Minimum volume ellipsoids - theory and algorithms , 2016, MOS-SIAM Series on Optimization.

[13]  Gil Zussman,et al.  A Fast Distributed Stateless Algorithm for alpha-Fair Packing Problems , 2015, ICALP.

[14]  Yin Tat Lee,et al.  Solving Linear Programs with Sqrt(rank) Linear System Solves , 2019, ArXiv.

[15]  R. Freund,et al.  Analysis of the Frank-Wolfe Method for Logarithmically-Homogeneous Barriers, with an Extension , 2020 .

[16]  Kevin A. Lai,et al.  $\ell_1$ Regression using Lewis Weights Preconditioning and Stochastic Gradient Descent , 2018, COLT.

[17]  Ruosong Wang,et al.  Nearly Linear Row Sampling Algorithm for Quantile Regression , 2020, ICML.

[18]  Richard Peng,et al.  Uniform Sampling for Matrix Approximation , 2014, ITCS.

[19]  Y. Lee.  Faster algorithms for convex and combinatorial optimization , 2016 .

[20]  David P. Woodruff,et al.  Strong Coresets for k-Median and Subspace Approximation: Goodbye Dimension , 2018, 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS).

[21]  Yin Tat Lee,et al.  A near-optimal algorithm for approximating the John Ellipsoid , 2019, COLT.

[22]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[23]  Yin Tat Lee,et al.  Strong self-concordance and sampling , 2019, STOC.

[24]  P. Wojtaszczyk.  Banach Spaces For Analysts: Preface , 1991 .

[25]  Pravin M. Vaidya,et al.  A new algorithm for minimizing convex functions over convex sets , 1996, Math. Program..

[26]  Mohit Singh,et al.  Combinatorial Algorithms for Optimal Design , 2019, COLT.

[27]  Man-Chung Yue,et al.  Universal Barrier Is n-Self-Concordant , 2018, Math. Oper. Res..

[28]  F. Pukelsheim.  Optimal Design of Experiments (Classics in Applied Mathematics) (Classics in Applied Mathematics, 50) , 2006 .

[29]  S. Muthukrishnan,et al.  Sampling algorithms for l2 regression and applications , 2006, SODA '06.

[30]  Mohit Singh,et al.  Approximation Algorithms for D-optimal Design , 2018, Math. Oper. Res..

[31]  Leonid Khachiyan,et al.  Rounding of Polytopes in the Real Number Model of Computation , 1996, Math. Oper. Res..

[32]  Richard Peng,et al.  Lp Row Sampling by Lewis Weights , 2015, STOC.

[33]  Piyush Kumar,et al.  Minimum-Volume Enclosing Ellipsoids and Core Sets , 2005 .

[34]  David P. Woodruff,et al.  Fast approximation of matrix coherence and statistical leverage , 2011, ICML.

[35]  Yin Tat Lee,et al.  Path Finding Methods for Linear Programming: Solving Linear Programs in Õ(vrank) Iterations and Faster Algorithms for Maximum Flow , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[36]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[37]  Gary L. Miller,et al.  Iterative Row Sampling , 2012, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[38]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[39]  Yin Tat Lee,et al.  A Faster Cutting Plane Method and its Implications for Combinatorial and Convex Optimization , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.