Learning parities in the mistake-bound model

We study the problem of learning parity functions that depend on at most $k$ variables ($k$-parities) attribute-efficiently in the mistake-bound model. We design a simple, deterministic, polynomial-time algorithm for learning $k$-parities with mistake bound $O(n^{1-frac{c}{k}})$, for any constant $c > 0$. This is the first polynomial-time algorithms that learns $omega(1)$-parities in the mistake-bound model with mistake bound $o(n)$. Using the standard conversion techniques from the mistake-bound model to the PAC model, our algorithm can also be used for learning $k$-parities in the PAC model. In particular, this implies a slight improvement on the results of Klivans and Servedio cite{rocco} for learning $k$-parities in the PAC model. We also show that the $widetilde{O}(n^{k/2})$ time algorithm from cite{rocco} that PAC-learns $k$-parities with optimal sample complexity can be extended to the mistake-bound model.

[1]  Larry J. Stockmeyer,et al.  The complexity of approximate counting , 1983, STOC.

[2]  Avrim Blum,et al.  Separating distribution-free and mistake-bound learning models over the Boolean domain , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[3]  Manuel Blum,et al.  Secure Human Identification Protocols , 2001, ASIACRYPT.

[4]  Nick Littlestone,et al.  From on-line to batch learning , 1989, COLT '89.

[5]  Avrim Blum,et al.  On-line Algorithms in Machine Learning , 1996, Online Algorithms.

[7]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[8]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[9]  Noga Alon,et al.  Deterministic Approximation Algorithms for the Nearest Codeword Problem , 2009, APPROX-RANDOM.

[10]  Rocco A. Servedio,et al.  Toward Attribute Efficient Learning of Decision Lists and Parities , 2006, J. Mach. Learn. Res..