On learning k-parities with and without noise

We first consider the problem of learning k-parities in the on-line mistake-bound model: given a hidden vector x ∈ {0,1} n with |x| = k and a sequence of “questions” a1,a2,··· ∈ {0,1} n , where the algorithm must reply to each question with hai,xi (mod 2), what is the best tradeoff between the number of mistakes made by the algorithm and its time complexity? We improve the previous best result of Buhrman et. al. [BGM10] by an exp(k) factor in the time complexity. Second, we consider the problem of learning k-parities in the presence of classification noise of rate � ∈ (0, 1/2). A polynomial time algorithm for this problem (when � > 0 and k = !(1)) is a longstanding challenge in learning theory. Grigorescu et al. [GRV11] showed an algorithm running in time n k/2

[1]  Rocco A. Servedio,et al.  Toward Attribute Efficient Learning of Decision Lists and Parities , 2006, J. Mach. Learn. Res..

[2]  Nick Littlestone,et al.  From on-line to batch learning , 1989, COLT '89.

[3]  Vitaly Feldman,et al.  On Agnostic Learning of Parities, Monomials, and Halfspaces , 2009, SIAM J. Comput..

[4]  Harry Buhrman,et al.  Learning parities in the mistake-bound model , 2010, Inf. Process. Lett..

[5]  Noga Alon,et al.  Deterministic Approximation Algorithms for the Nearest Codeword Problem , 2009, APPROX-RANDOM.

[6]  Santosh S. Vempala,et al.  On Noise-Tolerant Learning of Sparse Parities and Related Problems , 2011, ALT.

[7]  Avrim Blum,et al.  Separating distribution-free and mistake-bound learning models over the Boolean domain , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[8]  Gregory Valiant,et al.  Finding Correlations in Subquadratic Time, with Applications to Learning Parities and Juntas , 2012, 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science.

[9]  Vinod Vaikuntanathan,et al.  Efficient Fully Homomorphic Encryption from (Standard) LWE , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[10]  Oded Regev,et al.  On lattices, learning with errors, random linear codes, and cryptography , 2005, STOC '05.

[11]  Avrim Blum,et al.  On-line Algorithms in Machine Learning , 1996, Online Algorithms.

[12]  Manuel Blum,et al.  Secure Human Identification Protocols , 2001, ASIACRYPT.

[13]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[14]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.