Testing (Subclasses of) Halfspaces

We address the problem of testing whether a Boolean-valued function f is a halfspace, i.e. a function of the form f(x) = sgn(wċx-θ). We consider halfspaces over the continuous domain Rn (endowed with the standard multivariate Gaussian distribution) as well as halfspaces over the Boolean cube {-1, 1}n (endowed with the uniform distribution). In both cases we give an algorithm that distinguishes halfspaces from functions that are e-far from any halfspace using only poly(1/e) queries, independent of the dimension n. In contrast to the case of general halfspaces, we show that testing natural subclasses of halfspaces can be markedly harder; for the class of {-1, 1}-weight halfspaces, we show that a tester must make at least O(log n) queries. We complement this lower bound with an upper bound showing that O(√n queries suffice.

[1]  O. Svensson,et al.  Inapproximability Results for Sparsest Cut, Optimal Linear Arrangement, and Precedence Constrained Scheduling , 2007, FOCS 2007.

[2]  E. Fischer.  THE ART OF UNINFORMED DECISIONS: A PRIMER TO PROPERTY TESTING , 2004 .

[3]  Dana Ron,et al.  Property testing and its connection to learning and approximation , 1998, JACM.

[4]  Rocco A. Servedio,et al.  Testing Halfspaces , 2007, SIAM J. Comput..

[5]  Albert B Novikoff,et al.  ON CONVERGENCE PROOFS FOR PERCEPTRONS , 1963 .

[6]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[7]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[8]  Rocco A. Servedio,et al.  Testing for Concise Representations , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[9]  Guy Kindler,et al.  Testing juntas , 2002, J. Comput. Syst. Sci..

[10]  Pavel Pudlák,et al.  Threshold circuits of bounded depth , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[11]  Hans Ulrich Simon,et al.  On restricted-focus-of-attention learnability of Boolean functions , 1996, COLT '96.

[12]  Paul W. Goldberg,et al.  A Bound on the Precision Required to Estimate a Boolean Perceptron from Its Average Satisfying Assignment , 2006, SIAM J. Discret. Math..

[13]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[14]  Andrew Chi-Chih Yao,et al.  ON ACC and threshold circuits , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[15]  H. D. Block.  The perceptron: a model for brain functioning. I , 1962 .

[16]  John N. Tsitsiklis,et al.  Active Learning Using Arbitrary Binary Valued Queries , 1993, Machine Learning.

[17]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[18]  Rocco A. Servedio,et al.  Every Linear Threshold Function has a Low-Weight Approximator , 2006, 21st Annual IEEE Conference on Computational Complexity (CCC'06).

[19]  Lisa Hellerstein.  On generalized constraints and certificates , 2001, Discret. Math..

[20]  Ryan O'Donnell,et al.  Testing +/- 1-Weight Halfspaces , 2009 .

[21]  Ryan O'Donnell,et al.  Testing ±1-weight halfspace , 2009, APPROX-RANDOM.

[22]  Eldar Fischer,et al.  A Review of Graph Grammars and Preview of ICGT 2002: The First International Conference on Graph Transformation. , 2001 .

[23]  C. K. Chow,et al.  On the characterization of threshold functions , 1961, SWCT.