Learning DNF from random walks

We consider a model of learning Boolean functions from examples generated by a uniform random walk on {0, 1}/sup n/. We give a polynomial time algorithm for learning decision trees and DNF formulas in this model. This is the first efficient algorithm for learning these classes in a natural passive learning model where the learner has no influence over the choice of examples used for learning.

[1]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[2]  Rocco A. Servedio,et al.  Learning DNF in time 2Õ(n1/3) , 2004, J. Comput. Syst. Sci..

[3]  Karsten A. Verbeurgt.  Learning DNF under the uniform distribution in quasi-polynomial time , 1990, COLT '90.

[4]  I. Benjamini,et al.  Noise sensitivity of Boolean functions and applications to percolation , 1998 .

[5]  David Gamarnik.  Extension of the PAC framework to finite and countable Markov chains , 2003, IEEE Trans. Inf. Theory.

[6]  Michael Kharitonov,et al.  Cryptographic hardness of distribution-specific learning , 1993, STOC.

[7]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[8]  Richard J. Lipton,et al.  Amplification of weak learning under the uniform distribution , 1993, COLT '93.

[9]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1989, 30th Annual Symposium on Foundations of Computer Science.

[10]  Roni Khardon.  On Using the Fourier Transform to Learn Disjoint DNF , 1994, Inf. Process. Lett..

[11]  Ryan O'Donnell,et al.  Hardness amplification within NP , 2002, Proceedings 17th IEEE Annual Conference on Computational Complexity.

[12]  David Haussler,et al.  Learning decision trees from random examples , 1988, COLT '88.

[13]  Rocco A. Servedio,et al.  On learning monotone DNF under product distributions , 2001, Inf. Comput..

[14]  Rocco A. Servedio,et al.  On PAC learning using Winnow, Perceptron, and a Perceptron-like algorithm , 1999, COLT '99.

[15]  Leonid A. Levin,et al.  A hard-core predicate for all one-way functions , 1989, STOC '89.

[16]  Karsten A. Verbeurgt.  Learning Sub-classes of Monotone DNF on the Uniform Distribution , 1998, ALT.

[17]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[18]  Yishay Mansour,et al.  An O(nlog log n) learning algorithm for DNF under the uniform distribution , 1992, COLT '92.

[19]  Pavel Pudlák,et al.  On the Computational Power of Depth-2 Circuits with Threshold and Modulo Gates , 1997, Theor. Comput. Sci..

[20]  Nader H. Bshouty,et al.  More efficient PAC-learning of DNF with membership queries under the uniform distribution , 2004, J. Comput. Syst. Sci..

[21]  Rocco A. Servedio,et al.  Boosting and hard-core sets , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[22]  Eyal Kushilevitz,et al.  Learning decision trees using the Fourier spectrum , 1991, STOC '91.

[23]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[24]  Peter L. Bartlett,et al.  Exploiting Random Walks for Learning , 2002, Inf. Comput..

[25]  Nader H. Bshouty,et al.  On the Fourier spectrum of monotone functions , 1995, STOC '95.

[26]  Rocco A. Servedio,et al.  Learnability beyond AC0 , 2002, STOC '02.

[27]  Nader H. Bshouty,et al.  Learning DNF over the uniform distribution using a quantum example oracle , 1995, COLT '95.

[28]  Thomas Raysor Hancock.  The complexity of learning formulas and decision trees that have restricted reads , 1992 .

[29]  Umesh V. Vazirani,et al.  A Markovian extension of Valiant's learning model , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[30]  Yishay Mansour,et al.  An O(n^(log log n)) Learning Algorithm for DNT under the Uniform Distribution , 1995, J. Comput. Syst. Sci..

[31]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[32]  Adam R. Klivans,et al.  Learning DNF in time 2 Õ(n 1/3 ) . , 2001, STOC 2001.

[33]  Mihir Bellare.  A technique for upper bounding the spectral norm with applications to learning , 1992, COLT '92.

[34]  Vitaly Feldman,et al.  On Using Extended Statistical Queries to Avoid Membership Queries , 2001, J. Mach. Learn. Res..

[35]  Yishay Mansour,et al.  Weakly learning DNF and characterizing statistical query learning using Fourier analysis , 1994, STOC '94.

[36]  Jeffrey C. Jackson,et al.  An efficient membership-query algorithm for learning DNF with respect to the uniform distribution , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[37]  Nader H. Bshouty,et al.  Uniform-distribution attribute noise learnability , 1999, COLT '99.

[38]  Akira Maruoka,et al.  Learning Monotone Log-Term DNF Formulas under the Uniform Distribution , 2000, Theory of Computing Systems.

[39]  Yishay Mansour,et al.  Learning monotone ku DNF formulas on product distributions , 1991, COLT '91.

[40]  Leonid A. Levin,et al.  A Pseudorandom Generator from any One-way Function , 1999, SIAM J. Comput..