Simple learning algorithms for decision trees and multivariate polynomials

In this paper we develop a new approach for learning decision trees and multivariate polynomials via interpolation of multivariate polynomials. This new approach yields simple learning algorithms for multivariate polynomials and decision trees over finite fields under any constant bounded product distribution. The output hypothesis is a (single) multivariate polynomial that is an /spl epsiv/-approximation of the target under any constant bounded product distribution. The new approach demonstrates the learnability of many classes under any constant bounded product distribution and using membership queries, such as j-disjoint DNF and multivariate polynomial with bounded degree over any field. The technique shows how to interpolate multivariate polynomials with bounded term size from membership queries only. This in particular gives a learning algorithm for O(log n)-depth decision tree from membership queries only and a new learning algorithm of any multivariate polynomial over sufficiently large fields from membership queries only. We show that our results for learning from membership queries only are the best possible.

[1]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[2]  Ron M. Roth,et al.  Interpolation and Approximation of Sparse Multivariate Polynomials over GF(2) , 1991, SIAM J. Comput..

[3]  Marek Karpinski,et al.  Fast Parallel Algorithms for Sparse Multivariate Polynomial Interpolation over Finite Fields , 1988, SIAM J. Comput..

[4]  Tadao Kasami,et al.  A Polynomial Time Learning Algorithm for Recognizable Series , 1994 .

[5]  Michael Ben-Or,et al.  A deterministic algorithm for sparse multivariate polynomial interpolation , 1988, STOC '88.

[6]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[7]  Eyal Kushilevitz,et al.  Learning Decision Trees Using the Fourier Spectrum , 1993, SIAM J. Comput..

[8]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[9]  Yishay Mansour.  Randomized Interpolation and Approximation of Sparse Polynomials , 1992, ICALP.

[10]  Linda Sellie,et al.  Learning sparse multivariate polynomials over a field with queries and counterexamples , 1993, COLT '93.

[11]  Nader H. Bshouty.  A note on learning multivariate polynomials under the uniform distribution (extended abstract) , 1995, COLT '95.

[12]  Yishay Mansour,et al.  Randomized Interpolation and Approximation of Sparse Polynomials , 1992, SIAM J. Comput..

[13]  Francesco Bergadano,et al.  Learning Sat-k-DNF formulas from membership queries , 1996, STOC '96.

[14]  Nader H. Bshouty.  Exact Learning Boolean Function via the Monotone Theory , 1995, Inf. Comput..

[15]  Francesco Bergadano,et al.  Learning Behaviors of Automata from Multiplicity and Equivalence Queries , 1996, SIAM J. Comput..

[16]  Nader H. Bshouty.  Simple learning algorithms using divide and conquer , 1995, COLT '95.

[17]  Roni Khardon.  On Using the Fourier Transform to Learn Disjoint DNF , 1994, Inf. Process. Lett..

[18]  Marek Karpinski,et al.  On Zero-Testing and Interpolation of k-Sparse Multivariate Polynomials Over Finite Fields , 1991, Theor. Comput. Sci..

[19]  Richard Zippel,et al.  Interpolating Polynomials from Their Values , 1990, J. Symb. Comput..

[20]  Ming-Deh A. Huang,et al.  Interpolation of sparse multivariate polynomials over large finite fields with applications , 1996, SODA '96.

[21]  Nader H. Bshouty.  On Learning Multivariate Polynomials Under the Uniform Distribution , 1997, Inf. Process. Lett..

[22]  Eyal Kushilevitz,et al.  On the applications of multiplicity automata in learning , 1996, Proceedings of 37th Conference on Foundations of Computer Science.