Complexity Lower Bounds using Linear Algebra

We survey several techniques for proving lower bounds in Boolean, algebraic, and communication complexity based on certain linear algebraic approaches. The common theme among these approaches is to study robustness measures of matrix rank that capture the complexity in a given model. Suitably strong lower bounds on such robustness functions of explicit matrices lead to important consequences in the corresponding circuit or communication models. Many of the linear algebraic problems arising from these approaches are independently interesting mathematical challenges.

[1]  Avi Wigderson,et al.  The Randomized Communication Complexity of Set Disjointness , 2007, Theory Comput..

[2]  Béla Bollobás,et al.  Random Graphs , 1985 .

[3]  Alexander A. Sherstov,et al.  The Sign-rank of Ac , 2008 .

[4]  Murali K. Ganapathy,et al.  On the number of zero-patterns of a sequence of polynomials , 2001 .

[5]  Janos Simon,et al.  Probabilistic Communication Complexity , 1986, J. Comput. Syst. Sci..

[6]  Alexander A. Razborov,et al.  Applications of matrix methods to the theory of lower bounds in computational complexity , 1990, Comb..

[7]  Walter Baur,et al.  Simplified Lower Bounds for Polynomials with Algebraic Coefficients , 1997, J. Complex..

[8]  Jun Tarui.  Probablistic Polynomials, AC0 Functions, and the Polynomial-Time Hierarchy , 1993, Theor. Comput. Sci..

[9]  Shai Ben-David,et al.  Limitations of Learning Via Embeddings in Euclidean Half Spaces , 2003, J. Mach. Learn. Res..

[10]  A. Besicovitch,et al.  On the Linear Independence of Fractional Powers of Integers , 1940 .

[11]  Andrew Chi-Chih Yao,et al.  Some complexity questions related to distributive computing(Preliminary Report) , 1979, STOC.

[12]  Alexander A. Razborov,et al.  The Sign-Rank of AC^O , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[13]  Leslie G. Valiant,et al.  Graph-Theoretic Arguments in Low-Level Complexity , 1977, MFCS.

[14]  T. Tao.  An uncertainty principle for cyclic groups of prime order , 2003, math/0308286.

[15]  Peter Bürgisser,et al.  Lower bounds on the bounded coefficient complexity of bilinear maps , 2004, JACM.

[16]  Jürgen Forster.  A linear lower bound on the unbounded error probabilistic communication complexity , 2002, J. Comput. Syst. Sci..

[17]  Hartmut Klauck.  Lower Bounds for Quantum Communication Complexity , 2007, SIAM J. Comput..

[18]  J. W. Silverstein.  The Smallest Eigenvalue of a Large Dimensional Wishart Matrix , 1985 .

[19]  G. Pólya,et al.  Problems and theorems in analysis , 1983 .

[20]  Joel Friedman,et al.  A note on matrix rigidity , 1993, Comb..

[21]  Noam Nisan,et al.  Lower bounds on arithmetic circuits via partial derivatives , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[22]  A. S.  The Pattern Matrix Method for Lower Bounds on Quantum Communication ∗ , 2007 .

[23]  Noam Nisan,et al.  On rank vs. communication complexity , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[24]  Daniel A. Spielman,et al.  A Remark on Matrix Rigidity , 1997, Inf. Process. Lett..

[25]  Enav Weinreb,et al.  Separating the power of monotone span programs over different fields , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[26]  Noam Nisan,et al.  On the degree of boolean functions as real polynomials , 1992, STOC '92.

[27]  László Lovász,et al.  On the Shannon capacity of a graph , 1979, IEEE Trans. Inf. Theory.

[28]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[29]  Satyanarayana V. Lokam,et al.  Relations Between Communication Complexity, Linear Arrangements, and Computational Complexity , 2001, FSTTCS.

[30]  P. Frankl,et al.  Linear Algebra Methods in Combinatorics I , 1988 .

[31]  Michael E. Saks,et al.  Communication Complexity and Combinatorial Lattice Theory , 1993, J. Comput. Syst. Sci..

[32]  Matthias Krause,et al.  Geometric arguments yield better bounds for threshold circuits and distributed computing , 1991, [1991] Proceedings of the Sixth Annual Structure in Complexity Theory Conference.

[33]  Nathan Linial,et al.  Learning Complexity vs. Communication Complexity , 2008, 2008 23rd Annual IEEE Conference on Computational Complexity.

[34]  Noam Nisan,et al.  On the Complexity of Bilinear Forms , 2002 .

[35]  Jin-Yi Cai,et al.  On testing for zero polynomials by a set of points with bounded precision , 2001, Theor. Comput. Sci..

[36]  Santosh S. Vempala,et al.  An algorithmic theory of learning: Robust concepts and random projection , 1999, Machine Learning.

[37]  Kurt Mehlhorn,et al.  Las Vegas is better than determinism in VLSI and distributed computing (Extended Abstract) , 1982, STOC '82.

[38]  Nathan Linial,et al.  Lower bounds in communication complexity based on factorization norms , 2007, STOC '07.

[39]  P. Stevenhagen,et al.  Chebotarëv and his density theorem , 1996 .

[40]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[41]  Avi Wigderson,et al.  On span programs , 1993, [1993] Proceedings of the Eigth Annual Structure in Complexity Theory Conference.

[42]  Peter Frankl,et al.  Complexity classes in communication complexity theory , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[43]  Rüdiger Reischuk,et al.  Relations between Communication Complexity Classes , 1990, J. Comput. Syst. Sci..

[44]  Jun Tarui.  Randomized Polynomials, Threshold Circuits, and the Polynomial Hierarchy , 1991, STACS.

[45]  Eyal Kushilevitz,et al.  On the complexity of communication complexity , 2009, STOC '09.

[46]  Satyanarayana V. Lokam.  Graph Complexity and Slice Functions , 2002, Theory of Computing Systems.

[47]  Jacques Morgenstern,et al.  Note on a Lower Bound on the Linear Complexity of the Fast Fourier Transform , 1973, JACM.

[48]  Jehoshua Bruck,et al.  Polynomial threshold functions, AC functions and spectrum norms , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[49]  Vojtech Rödl,et al.  A combinatorial approach to complexity , 1992, Comb..

[50]  Pavel Pudlák,et al.  A note on the use of determinant for proving lower bounds on the size of linear circuits , 2000, Inf. Process. Lett..

[51]  Jaikumar Radhakrishnan,et al.  Bounds for Dispersers, Extractors, and Depth-Two Superconcentrators , 2000, SIAM J. Discret. Math..

[52]  Gatis Midrijanis,et al.  Three lines proof of the lower bound for the matrix rigidity , 2005, ArXiv.

[53]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[54]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[55]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[56]  Adam R. Klivans,et al.  Learning DNF in time 2 Õ(n 1/3 ) . , 2001, STOC 2001.

[57]  A. Hoffman,et al.  The variation of the spectrum of a normal matrix , 1953 .

[58]  Bernard Chazelle.  A spectral approach to lower bounds , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[59]  Anna Gál.  A characterization of span program size and improved lower bounds for monotone span programs , 1998, STOC '98.

[60]  Pavel Pudlák,et al.  On the Computational Power of Depth-2 Circuits with Threshold and Modulo Gates , 1997, Theor. Comput. Sci..

[61]  Nathan Linial,et al.  Complexity measures of sign matrices , 2007, Comb..

[62]  Alexander A. Razborov,et al.  Natural Proofs , 1997, J. Comput. Syst. Sci..

[63]  Srinivasan Venkatesh,et al.  Approximating the Radii of Point Sets , 2007, SIAM J. Comput..

[64]  Enav Weinreb,et al.  Separating the Power of Monotone Span Programs over Different Fields , 2005, SIAM J. Comput..

[65]  Vojtech Rödl,et al.  Some combinatorial-algebraic problems from complexity theory , 1994, Discret. Math..

[66]  Ran Raz,et al.  On the complexity of matrix product , 2002, STOC '02.

[67]  A. Wigderson.  The Fusion Method for Lower Bounds in Circuit Complexity , 2003 .

[68]  Santosh S. Vempala,et al.  The Colin de Verdière number and sphere representations of a graph , 1997, Comb..

[69]  Avi Wigderson,et al.  Superpolynomial Lower Bounds for Monotone Span Programs , 1996, Comb..

[70]  László Lovász,et al.  On the ratio of optimal integral and fractional covers , 1975, Discret. Math..

[71]  Béla Bollobás,et al.  Modern Graph Theory , 2002, Graduate Texts in Mathematics.

[72]  Satyanarayana V. Lokam.  Quadratic Lower Bounds on Matrix Rigidity , 2006, TAMC.

[73]  Vojtech Rödl,et al.  Modified ranks of tensors and the size of circuits , 1993, STOC '93.

[74]  H. Warren.  Lower bounds for approximation by nonlinear manifolds , 1968 .

[75]  Ronald de Wolf.  Lower Bounds on Matrix Rigidity Via a Quantum Argument , 2006, ICALP.

[76]  Ron C. Blei.  An elementary proof of the Grothendieck inequality , 1987 .

[77]  Vojtech Rödl,et al.  Boolean Circuits, Tensor Ranks, and Communication Complexity , 1997, SIAM J. Comput..

[78]  Michael Alekhnovich.  More on Average Case vs Approximation Complexity , 2011, computational complexity.

[79]  Noam Nisan,et al.  On the complexity of bilinear forms: dedicated to the memory of Jacques Morgenstern , 1995, STOC '95.

[80]  Salil P. Vadhan,et al.  Checking polynomial identities over any field: towards a derandomization? , 1998, STOC '98.

[81]  Vojtech Rödl,et al.  Graph complexity , 1988, Acta Informatica.

[82]  Pavel Pudlák,et al.  Circuit lower bounds and linear codes , 2006, Electron. Colloquium Comput. Complex..

[83]  Volker Strassen,et al.  Algebraic Complexity Theory , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[84]  Gene H. Golub,et al.  Matrix computations , 1983 .

[85]  P. Erdoes,et al.  On sparse graphs with dense long paths. , 1975 .

[86]  A. Razborov,et al.  Improved lower bounds on the rigidity of Hadamard matrices , 1998 .

[87]  Victor Shoup,et al.  Lower bounds for polynomial evaluation and interpolation problems , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[88]  Walter Baur,et al.  The Complexity of Partial Derivatives , 1983, Theor. Comput. Sci..

[89]  Pavel Pudlák,et al.  Communication in bounded depth circuits , 1994, Comb..

[90]  Satyanarayana V. Lokam.  On the rigidity of Vandermonde matrices , 2000, Theor. Comput. Sci..

[91]  Vojtech Rödl,et al.  Geometrical realization of set systems and probabilistic communication complexity , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[92]  Zhi-Zhong Chen,et al.  Reducing randomness via irrational numbers , 1997, STOC '97.

[93]  Hans Ulrich Simon,et al.  On the smallest possible dimension and the largest possible margin of linear arrangements representing given concept classes , 2006, Theor. Comput. Sci..

[94]  Anna Gál,et al.  Lower bounds for monotone span programs , 2005, computational complexity.

[95]  Joachim von zur Gathen,et al.  Modern Computer Algebra , 1998 .

[96]  E. Kushilevitz,et al.  Communication Complexity: Basics , 1996 .

[97]  Satyanarayana V. Lokam.  Spectral Methods for Matrix Rigidity with Applications to Size-Depth Trade-offs and Communication Complexity , 2001, J. Comput. Syst. Sci..

[98]  Beate Bollig,et al.  On the Power of Different Types of Restricted Branching Programs , 1994, Electron. Colloquium Comput. Complex..

[99]  Avi Wigderson,et al.  Monotone Circuits for Connectivity Require Super-Logarithmic Depth , 1990, SIAM J. Discret. Math..

[100]  Rocco A. Servedio,et al.  Learning DNF in time 2Õ(n1/3) , 2004, J. Comput. Syst. Sci..