Testing Halfspaces

This paper addresses the problem of testing whether a Boolean-valued function f is a halfspace, i.e. a function of the form f(x) = sgn(w · x - θ). We consider halfspaces over the continuous domain Rn (endowed with the standard multivariate Gaussian distribution) as well as halfspaces over the Boolean cube {−1, 1}n (endowed with the uniform distribution). In both cases we give an algorithm that distinguishes halfspaces from functions that are e-far from any halfspace using only poly(1/e) queries, independent of the dimension n. Two simple structural results about halfspaces are at the heart of our approach for the Gaussian distribution: the first gives an exact relationship between the expected value of a halfspace f and the sum of the squares of f's degree-1 Hermite coefficients, and the second shows that any function that approximately satisfies this relationship is close to a halfspace. We prove analogous results for the Boolean cube {−1, 1}n (with Fourier coefficients in place of Hermite coefficients) for balanced halfspaces in which all degree-1 Fourier coefficients are small. Dealing with general halfspaces over {−1, 1}n poses significant additional complications and requires other ingredients. These include "cross-consistency" versions of the results mentioned above for pairs of halfspaces with the same weights but different thresholds; new structural results relating the largest degree-1 Fourier coefficient and the largest weight in unbalanced halfspaces; and algorithmic techniques from recent work on testing juntas [FKR+02].

[1]  Noga Alon,et al.  Testing Low-Degree Polynomials over GF(2( , 2003, RANDOM-APPROX.

[2]  Michel Talagrand,et al.  How much are increasing sets positively correlated? , 1996, Comb..

[3]  Paul W. Goldberg,et al.  A Bound on the Precision Required to Estimate a Boolean Perceptron from Its Average Satisfying Assignment , 2006, SIAM J. Discret. Math..

[4]  Paul W. Goldberg,et al.  Estimating a Boolean Perceptron from Its Average Satisfying Assignment: A Bound on the Precision Required , 2001, COLT/EuroCOLT.

[5]  Dana Ron Property Testing: A Learning Theory Perspective , 2008, Found. Trends Mach. Learn..

[6]  C. K. Chow,et al.  On the characterization of threshold functions , 1961, SWCT.

[7]  R. Rao,et al.  Normal Approximation and Asymptotic Expansions , 1976 .

[8]  Yuval Rabani,et al.  Linear Programming , 2007, Handbook of Approximation Algorithms and Metaheuristics.

[9]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[10]  Manuel Blum,et al.  Self-testing/correcting with applications to numerical problems , 1990, STOC '90.

[11]  Rocco A. Servedio,et al.  Testing for Concise Representations , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[12]  Ryan O'Donnell,et al.  The chow parameters problem , 2008, SIAM J. Comput..

[13]  Albert B Novikoff,et al.  ON CONVERGENCE PROOFS FOR PERCEPTRONS , 1963 .

[14]  Guy Kindler,et al.  Optimal inapproximability results for MAX-CUT and other 2-variable CSPs? , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[15]  Guy Kindler,et al.  Testing juntas , 2002, J. Comput. Syst. Sci..

[16]  J. Kuelbs Probability on Banach spaces , 1978 .

[17]  Dana Ron,et al.  Testing Monotonicity , 2000, Comb..

[18]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[19]  Ronitt Rubinfeld,et al.  Monotonicity testing over general poset domains , 2002, STOC '02.

[20]  Rocco A. Servedio,et al.  Every Linear Threshold Function has a Low-Weight Approximator , 2006, 21st Annual IEEE Conference on Computational Complexity (CCC'06).

[21]  Dana Ron,et al.  Property testing and its connection to learning and approximation , 1998, JACM.

[22]  Hans Ulrich Simon,et al.  On restricted-focus-of-attention learnability of Boolean functions , 1996, COLT '96.

[23]  Dana Ron,et al.  Testing Basic Boolean Formulae , 2002, SIAM J. Discret. Math..

[24]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[25]  Amos Fiat,et al.  Decision Trees: More Theoretical Justification for Practical Algorithms , 2004, ALT.

[26]  Andrew Chi-Chih Yao,et al.  ON ACC and threshold circuits , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[27]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[28]  H. D. Block The perceptron: a model for brain functioning. I , 1962 .

[29]  John N. Tsitsiklis,et al.  Active Learning Using Arbitrary Binary Valued Queries , 1993, Machine Learning.