Learning juntas

We consider a fundamental problem in computational learning theory: learning an arbitrary Boolean function which depends on an unknown set of <i>k</i> out of <i>n</i> Boolean variables. We give an algorithm for learning such functions from uniform random examples which runs in time roughly <i>(n<sup>k</sup>)<sup>ω/(ω + 1)</sup>,</i> where <i>ω < 2.376</i> is the matrix multiplication exponent. We thus obtain the first polynomial factor improvement on the naive <i>n<sup>k</sup></i> time bound which can be achieved via exhaustive search. Our algorithm and analysis exploit new structural properties of Boolean functions.

[1]  Avrim Blum,et al.  Relevant Examples and Relevant Features: Thoughts from Computational Learning Theory , 1994 .

[2]  Manfred K. Warmuth,et al.  Learning integer lattices , 1990, COLT '90.

[3]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[4]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1989, 30th Annual Symposium on Foundations of Computer Science.

[5]  Yishay Mansour,et al.  An O(n^(log log n)) Learning Algorithm for DNT under the Uniform Distribution , 1995, J. Comput. Syst. Sci..

[6]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1993, JACM.

[7]  Karsten A. Verbeurgt.  Learning Sub-classes of Monotone DNF on the Uniform Distribution , 1998, ALT.

[8]  Yishay Mansour,et al.  An O(nlog log n) learning algorithm for DNF under the uniform distribution , 1992, COLT '92.

[9]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[10]  Karsten A. Verbeurgt.  Learning DNF under the uniform distribution in quasi-polynomial time , 1990, COLT '90.

[11]  Yishay Mansour,et al.  Learning Boolean Functions via the Fourier Transform , 1994 .

[12]  Marco Protasi,et al.  On Learning Monotone DNF Formulae under Uniform Distributions , 1994, Inf. Comput..

[13]  Jeffrey C. Jackson,et al.  An efficient membership-query algorithm for learning DNF with respect to the uniform distribution , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[14]  Nader H. Bshouty,et al.  More efficient PAC-learning of DNF with membership queries under the uniform distribution , 2004, J. Comput. Syst. Sci..

[15]  Joachim von zur Gathen,et al.  Polynomials with two values , 1997, Comb..

[16]  Guy Kindler,et al.  Testing juntas , 2002, J. Comput. Syst. Sci..

[17]  Rocco A. Servedio,et al.  Learnability beyond AC0 , 2002, STOC '02.

[18]  Vitaly Feldman,et al.  On using extended statistical queries to avoid membership queries , 2002 .

[19]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[20]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[21]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[22]  Anna Bernasconi.  On a hierarchy of Boolean functions hard to compute in constant depth , 2001, Discret. Math. Theor. Comput. Sci..