On Learning Monotone DNF under Product Distributions

We show that the class of monotone 2O(√ n)-term DNF formulae can be PAC learned in polynomial time under the uniform distribution from random examples only. This is an exponential improvement over the best previous polynomial-time algorithms in this model, which could learn monotone o(log2 n)-term DNF, and is the first efficient algorithm for monotone (log n)ω(1)-term DNF in any nontrivial model of learning from random examples. We also show that various classes of small constant-depth circuits which compute monotone functions on few input variables are PAC learnable in polynomial time under the uniform distribution. All of our results extend to learning under any constantbounded product distribution.

[1]  Thomas Raysor Hancock.  The complexity of learning formulas and decision trees that have restricted reads , 1992 .

[2]  Leslie G. Valiant,et al.  Learning Boolean formulas , 1994, JACM.

[3]  Yuri Gurevich,et al.  Monotone versus positive , 1987, JACM.

[4]  Yishay Mansour,et al.  Weakly learning DNF and characterizing statistical query learning using Fourier analysis , 1994, STOC '94.

[5]  Marco Protasi,et al.  On Learning Monotone DNF Formulae under Uniform Distributions , 1994, Inf. Comput..

[6]  Eyal Kushilevitz,et al.  On the applications of multiplicity automata in learning , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[7]  Nader H. Bshouty,et al.  On the Fourier spectrum of monotone functions , 1996, JACM.

[8]  Ravi B. Boppana,et al.  The Complexity of Finite Functions , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[9]  Rocco A. Servedio,et al.  Boosting and hard-core sets , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[10]  Torben Hagerup,et al.  A Guided Tour of Chernoff Bounds , 1990, Inf. Process. Lett..

[11]  Jeffrey C. Jackson.  An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution , 1997, J. Comput. Syst. Sci..

[12]  Sean W. Smith,et al.  Improved learning of AC0 functions , 1991, COLT '91.

[13]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1993, JACM.

[14]  Nader H. Bshouty,et al.  More efficient PAC-learning of DNF with membership queries under the uniform distribution , 1999, COLT '99.

[15]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[16]  Avrim Blum,et al.  Fast learning of k-term DNF formulas with queries , 1992, STOC '92.

[17]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[18]  Karsten A. Verbeurgt.  Learning Sub-classes of Monotone DNF on the Uniform Distribution , 1998, ALT.

[19]  Akira Maruoka,et al.  Learning Monotone Log-Term DNF Formulas under the Uniform Distribution , 2000, Theory of Computing Systems.

[20]  Yishay Mansour,et al.  Learning monotone ku DNF formulas on product distributions , 1991, COLT '91.

[21]  Nathan Linial,et al.  The influence of variables on Boolean functions , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[22]  Nader H. Bshouty.  Exact Learning Boolean Function via the Monotone Theory , 1995, Inf. Comput..

[23]  Johan Håstad,et al.  Almost optimal lower bounds for small depth circuits , 1986, STOC '86.

[24]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[25]  Karsten A. Verbeurgt.  Learning DNF under the uniform distribution in quasi-polynomial time , 1990, COLT '90.

[26]  Mihir Bellare.  A technique for upper bounding the spectral norm with applications to learning , 1992, COLT '92.

[27]  J. Håstad.  Computational limitations of small-depth circuits , 1987 .

[28]  Akira Maruoka,et al.  Learning monotone log-term DNF formulas , 1994, COLT '94.

[29]  Roni Khardon.  On Using the Fourier Transform to Learn Disjoint DNF , 1994, Inf. Process. Lett..

[30]  Eyal Kushilevitz,et al.  A Simple Algorithm for Learning O (log n)-Term DNF , 1997, Inf. Process. Lett..

[31]  Yishay Mansour,et al.  An O(nlog log n) learning algorithm for DNF under the uniform distribution , 1992, COLT '92.