Learning the Fourier spectrum of probabilistic lists and trees

We observe that the Linial, Mansour, and Nissan method of learning boolean concepts (under uniform sampling distribution) by reconstructing their Fourier represent ation [LMN89] extends when the concepts are probabilistic in the sense of Kearns and Shapire [KS90]. We show that probabilistic decision lists, and more generally probabilistic decision trees with at most one occurrence of each literal, can be approximate ed by polynomially small Fourier represent ations, and that the non-negligible Fourier coefficients can be efficiently identified and estimated. Hence, all such concepts are learnable in polynomial time under uniform sampling distribution. This is the first instance where Fourier methods result in polynomial learning algorithms: the polynomiality of our results should be contrasted to the np”lylogn complexities in the analogous cases of [LMN89] and [M90]. The new ingredient of our work that allows us to achieve this polynomiality is that via refined Fourier analysis we are able to isolate the polynomially small set of non-negligible Fourier coefficients that reside in a super-polynomially large area of the spectrum. We further observe that several more general concept classes have slightly super-polynomial (npolyk)gn ) learning algorithms. These classes include all polynomial-size probabilistic decision trees, their convex combinations, etc. A concrete special case which results in polynomial learnabil“Bdl ColIl]lltl[\icalioI]s Research, Morristown NJ 07960. aidlo((!fl ash .Ixdlcorc.con]. flkll (bmmnnicat.ions Research, hlorrist.own NJ 07!w0, ]I~illail(@)fl&sll .l}cllcorc. col]). ity is the weighted arithmetization of k-DNF.