More efficient PAC-learning of DNF with membership queries under the uniform distribution

An efficient algorithm exists for learning disjunctive normal form (DNF) expressions in the uniformdistribution PAC learning model with membership queries (J. Comput. System Sci. 55 (1997) 414), but in practice the algorithm can only be applied to small problems. We present several modifications to the algorithm that substantially improve its asymptotic efficiency. First, we show how to significantly improve the time and sample complexity of a key subprogram, resulting in similar improvements in the bounds on the overall DNF algorithm. We also apply known methods to convert the resulting algorithm to an attribute efficient algorithm. Furthermore, we develop a technique for lower bounding the sample size required for PAC learning with membership queries under a fixed distribution and apply this technique to produce a lower bound on the number of membership queries needed for the uniform-distribution DNF learning problem. Finally, we present a learning algorithm for DNF that is attribute efficient in its use of random bits.

[1]  Moni Naor,et al.  Small-Bias Probability Spaces: Efficient Constructions and Applications , 1993, SIAM J. Comput..

[2]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[3]  Karsten A. Verbeurgt.  Learning DNF under the uniform distribution in quasi-polynomial time , 1990, COLT '90.

[4]  Lisa Hellerstein,et al.  Attribute-Efficient Learning in Query and Mistake-Bound Models , 1998, J. Comput. Syst. Sci..

[5]  Marek Karpinski,et al.  Learning read-once formulas with queries , 1993, JACM.

[6]  Leonid A. Levin,et al.  A hard-core predicate for all one-way functions , 1989, STOC '89.

[7]  Jeffrey C. Jackson.  An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution , 1997, J. Comput. Syst. Sci..

[8]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[9]  Yoav Freund,et al.  An improved boosting algorithm and its implications on learning complexity , 1992, COLT '92.

[10]  Oded Goldreich,et al.  Modern Cryptography, Probabilistic Proofs and Pseudorandomness , 1998, Algorithms and Combinatorics.

[11]  Rocco A. Servedio,et al.  Boosting and hard-core sets , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[12]  Jacobus H. van Lint,et al.  Introduction to Coding Theory , 1982 .

[13]  Eyal Kushilevitz,et al.  Learning Decision Trees Using the Fourier Spectrum , 1993, SIAM J. Comput..

[14]  Yishay Mansour,et al.  An O(nlog log n) learning algorithm for DNF under the uniform distribution , 1992, COLT '92.

[15]  NaorMoni,et al.  Small-bias probability spaces , 1993 .