Halfspace Matrices

A halfspace matrix is a Boolean matrix A with rows indexed by linear threshold functions f , columns indexed by inputs x isin {-1,1}n, and the entries given by Af ,x = f (x). We demonstrate the potential of halfspace matrices as tools to answer nontrivial open questions. (1) (Communication complexity) We exhibit a Boolean function f with discrepancy Omega(1/n4) under every product distribution but O(radicn /2n/4}) under a certain non-product distribution. This partially solves an open problem of Kushilevitz and Nisan. (2) (Complexity of sign matrices) We construct a matrix A isin {-1,1}NtimesN logN with dimension complexity logN but margin complexity Omega(N1/4/radic{log N}). This gap is an exponential improvement over previous work. As an application to circuit complexity, we prove an Omega(2n/4/(dradicn)) circuit lower bound for computing halfspaces by a majority of an arbitrary set of d gates. This complements a result of Goldmann, Hastad, and Razborov. In addition, we prove new results on the complexity measures of sign matrices, complementing recent work by Linial et al.(3) (Learning theory) We give a short and simple proof that the statistical-query (SQ) dimension of halfspaces in n dimensions is less than 2(n+1)2 under all distributions (with n+1 being a trivial lower bound). This improves on the nO(1) estimate from the fundamental paper of Blum et al. Finally, we motivate our learning-theoretic result for the complexity community by showing that SQ dimension estimates for natural classes of Boolean functions can resolve major open problems in complexity theory. Specifically, we show that an exp(2(logn) o(1) ) upper bound on the SQ dimension of AC0 would imply an explicit language in PSPACEcc\PHcc.

[1]  Janos Simon,et al.  Probabilistic Communication Complexity , 1986, J. Comput. Syst. Sci..

[2]  Alexander A. Razborov,et al.  Majority gates vs. general weighted threshold gates , 2005, computational complexity.

[3]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[4]  Hans Ulrich Simon.  Spectral Norm in Learning Theory: Some Selected Topics , 2006, ALT.

[5]  Peter Frankl,et al.  Complexity classes in communication complexity theory , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[6]  Bala Kalyanasundaram,et al.  The Probabilistic Communication Complexity of Set Intersection , 1992, SIAM J. Discret. Math..

[7]  Jürgen Forster.  A linear lower bound on the unbounded error probabilistic communication complexity , 2002, J. Comput. Syst. Sci..

[8]  Johan Håstad,et al.  On the Size of Weights for Threshold Gates , 1994, SIAM J. Discret. Math..

[9]  Yishay Mansour,et al.  Weakly learning DNF and characterizing statistical query learning using Fourier analysis , 1994, STOC '94.

[10]  Alexander A. Sherstov,et al.  Cryptographic Hardness for Learning Intersections of Halfspaces , 2006, FOCS.

[11]  M. Saks.  Slicing the hypercube , 1993 .

[12]  Jehoshua Bruck,et al.  On the Power of Threshold Circuits with Small Weights , 1991, SIAM J. Discret. Math..

[13]  Satyanarayana V. Lokam.  Spectral Methods for Matrix Rigidity with Applications to Size-Depth Trade-offs and Communication Complexity , 2001, J. Comput. Syst. Sci..

[14]  Alexander A. Razborov,et al.  On the Distributional Complexity of Disjointness , 1992, Theor. Comput. Sci..

[15]  Santosh S. Vempala,et al.  A random sampling based algorithm for learning the intersection of half-spaces , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[16]  Noam Nisan,et al.  On Randomized One-round Communication Complexity , 1995, STOC '95.

[17]  Rocco A. Servedio,et al.  Learning intersections and thresholds of halfspaces , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[18]  Ke Yang,et al.  New lower bounds for statistical query learning , 2002, J. Comput. Syst. Sci..

[19]  Nathan Linial,et al.  Complexity measures of sign matrices , 2007, Comb..

[20]  Hans Ulrich Simon,et al.  On the smallest possible dimension and the largest possible margin of linear arrangements representing given concept classes , 2006, Theor. Comput. Sci..

[21]  Stephen Kwek,et al.  PAC Learning Intersections of Halfspaces with Membership Queries , 1998, Algorithmica.

[22]  Alan M. Frieze,et al.  A Polynomial-Time Algorithm for Learning Noisy Linear Threshold Functions , 1996, Algorithmica.

[23]  Ran Raz,et al.  The BNS-Chung criterion for multi-party communication complexity , 2000, computational complexity.

[24]  Nikolai K. Vereshchagin,et al.  On Computation and Communication with Small Bias , 2007, Computational Complexity Conference.

[25]  Jeff Ford,et al.  Hadamard Tensors and Lower Bounds on Multiparty Communication Complexity , 2005, ICALP.

[26]  Hans Ulrich Simon,et al.  Estimating the Optimal Margins of Embeddings in Euclidean Half Spaces , 2004, Machine Learning.

[27]  Alexander A. Sherstov,et al.  Improved Lower Bounds for Learning Intersections of Halfspaces , 2006, COLT.

[28]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[29]  Vojtech Rödl,et al.  Geometrical realization of set systems and probabilistic communication complexity , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).