On Learning Random DNF Formulas Under the Uniform Distribution

We study the average-case learnability of DNF formulas in the model of learning from uniformly distributed random examples. We define a natural model of random monotone DNF formulas and give an efficient algorithm which with high probability can learn, for any fixed constant γ>0, a random t-term monotone DNF for any t = O(n2−γ). We also define a model of random nonmonotone DNF and give an efficient algorithm which with high probability can learn a random t-term DNF for any t=O(n3/2−γ). These are the first known algorithms that can successfully learn a broad class of polynomial-size DNF in a reasonable average-case model of learning from random examples.

[1]  Rocco A. Servedio,et al.  Learning Random Log-Depth Decision Trees under Uniform Distribution , 2005, SIAM J. Comput..

[2]  Leonard Pitt,et al.  On the learnability of disjunctive normal form formulas , 2004, Machine Learning.

[3]  D. Angluin.  Queries and Concept Learning , 2004, Machine Learning.

[4]  Avrim Blum,et al.  Machine learning: my favorite results, directions, and open problems , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[5]  Avrim Blum.  Learning a Function of r Relevant Variables , 2003, COLT.

[6]  Rocco A. Servedio,et al.  Learning Random Log-Depth Decision Trees under the Uniform Distribution , 2003, COLT.

[7]  Rocco A. Servedio,et al.  Learning intersections and thresholds of halfspaces , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[8]  Adam R. Klivans,et al.  Learnability Beyond AC 0 , 2002 .

[9]  Rocco A. Servedio,et al.  Smooth Boosting and Learning with Malicious Noise , 2001, J. Mach. Learn. Res..

[10]  Rocco A. Servedio,et al.  On learning monotone DNF under product distributions , 2001, Inf. Comput..

[11]  John Langford,et al.  On learning monotone Boolean functions , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[12]  Mario Marchand,et al.  On learning ?-perceptron networks on the uniform distribution , 1996, Neural Networks.

[13]  Jeffrey C. Jackson,et al.  An efficient membership-query algorithm for learning DNF with respect to the uniform distribution , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[14]  Yishay Mansour,et al.  Weakly learning DNF and characterizing statistical query learning using Fourier analysis , 1994, STOC '94.

[15]  Marco Protasi,et al.  On Learning Monotone DNF Formulae under Uniform Distributions , 1994, Inf. Comput..

[16]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[17]  Yishay Mansour,et al.  Learning Boolean Functions via the Fourier Transform , 1994 .

[18]  Thomas R. Hancock.  Learning kμ decision trees on the uniform distribution , 1993, COLT '93.

[19]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[20]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[21]  Dana Angluin,et al.  When won't membership queries help? , 1991, STOC '91.

[22]  Karsten A. Verbeurgt.  Learning DNF under the uniform distribution in quasi-polynomial time , 1990, COLT '90.

[23]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[24]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[25]  M. Kearns,et al.  Recent Results on Boolean Concept Learning , 1987 .

[26]  B. Bollobás.  Combinatorics: Set Systems, Hypergraphs, Families of Vectors and Combinatorial Probability , 1986 .

[27]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.