Hardness of Reconstructing Multivariate Polynomials over Finite Fields

We study the polynomial reconstruction problem, for low-degree multivariate polynomials over F[2]. In this problem, we are given a set of points x epsi {0, 1}n and target values f(x) epsi {0, 1} for each of these points, with the promise that there is a polynomial over F[2] of degree at most d that agrees with f at 1 - epsiv fraction of the points. Our goal is to find agree d polynomial that has good-agreement with f. We show that it is NP-hard to find a polynomial that agrees with f on more than 1 - 2-d + delta fraction of the points for any epsiv, delta > 0. This holds even with the stronger promise that the polynomial that fits the data is in fact linear, wherejis the algorithm is allowed to find a polynomial of degree d. Previously the only known, hardness of approximation (or even NP-completeness) was for the case when d = I, which follows from a celebrated result of Has tad. In the setting of computational learning, our result shows the hardness of (non-proper) agnostic learning of parities, where the learner is allowed, a low-degree polynomial over F[2] as a hypothesis. This is the first non-proper hardness result for this central problem in computational learning. Our results extend-to multivariate polynomial reconstruction over any finite field.

[1]  Amnon Ta-Shma,et al.  Extractors from Reed-Muller Codes , 2001, Electron. Colloquium Comput. Complex..

[2]  Ran Raz.  A Parallel Repetition Theorem , 1998, SIAM J. Comput..

[3]  Guy Kindler,et al.  On non-approximability for quadratic programs , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[4]  Dave Bacon,et al.  Classical simulation of quantum entanglement without local hidden variables , 2001 .

[5]  Sophie Laplante,et al.  Simulating quantum correlations as a distributed sampling problem (9 pages) , 2005, quant-ph/0507120.

[6]  Uri Zwick,et al.  A 7/8-approximation algorithm for MAX 3SAT? , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[7]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[8]  Vitaly Feldman,et al.  New Results for Learning Noisy Parities and Halfspaces , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[9]  Sanjeev Arora,et al.  Probabilistic checking of proofs: a new characterization of NP , 1998, JACM.

[10]  Johan Håstad,et al.  Some optimal inapproximability results , 2001, JACM.

[11]  Alexander Vardy,et al.  Correcting errors beyond the Guruswami-Sudan radius in polynomial time , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[12]  Ryan O'Donnell,et al.  Optimal Inapproximability Results for MAX-CUT and Other 2-Variable CSPs? , 2007, SIAM J. Comput..

[13]  A. Shimony,et al.  Proposed Experiment to Test Local Hidden Variable Theories. , 1969 .

[14]  Venkatesan Guruswami.  8 List Decoding of Concatenated Codes , 2004 .

[15]  Moses Charikar,et al.  Maximizing quadratic programs: extending Grothendieck's inequality , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[16]  N. Alon,et al.  Quadratic forms on graphs , 2006 .

[17]  Luca Trevisan,et al.  Pseudorandom generators without the XOR lemma , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[18]  D. Bacon,et al.  Communication cost of simulating Bell correlations. , 2003, Physical review letters.

[19]  Subhash Khot,et al.  On hardness of learning intersection of two halfspaces , 2008, STOC '08.

[20]  Albert Einstein,et al.  Can Quantum-Mechanical Description of Physical Reality Be Considered Complete? , 1935 .

[21]  Sophie Laplante,et al.  Classical simulation of traceless binary observables on any bipartite quantum state , 2007 .

[22]  S. Pironio,et al.  Violations of Bell inequalities as lower bounds on the communication cost of nonlocal correlations , 2003, quant-ph/0304176.

[23]  D. Bacon,et al.  Bell inequalities with auxiliary communication. , 2002, Physical review letters.

[24]  Adam Tauman Kalai,et al.  On agnostic boosting and parity learning , 2008, STOC.

[25]  Emanuele Viola,et al.  Pseudorandom Bits for Polynomials , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[26]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1993, JACM.

[27]  N J Cerf,et al.  Simulating maximal quantum entanglement without communication. , 2005, Physical review letters.

[28]  Emanuele Viola.  The Sum of d Small-Bias Generators Fools Polynomials of Degree d , 2008, Computational Complexity Conference.

[29]  Rocco A. Servedio,et al.  Agnostically Learning Halfspaces , 2005, FOCS.

[30]  Carsten Lund,et al.  Proof verification and the hardness of approximation problems , 1998, JACM.

[31]  Leonid A. Levin,et al.  A hard-core predicate for all one-way functions , 1989, STOC '89.

[32]  Venkatesan Guruswami,et al.  Maximum-likelihood decoding of Reed-Solomon codes is NP-hard , 2004, SODA.

[33]  J. Krivine.  Constantes de Grothendieck et fonctions de type positif sur les sphères , 1979 .

[34]  Venkatesan Guruswami,et al.  Explicit capacity-achieving list-decodable codes , 2005, STOC.

[35]  Roman Smolensky,et al.  Algebraic methods in the theory of lower bounds for Boolean circuit complexity , 1987, STOC.

[36]  Vitaly Feldman.  Optimal hardness results for maximizing agreements with monomials , 2006, 21st Annual IEEE Conference on Computational Complexity (CCC'06).

[37]  Qi Cheng,et al.  On the List and Bounded Distance Decodibility of the Reed-Solomon Codes (Extended Abstract) , 2004, FOCS.

[38]  I. J. Schoenberg.  Positive definite functions on spheres , 1942 .

[39]  Madhu Sudan,et al.  Decoding of Reed Solomon Codes beyond the Error-Correction Bound , 1997, J. Complex..

[40]  Gilles Brassard,et al.  Cost of Exactly Simulating Quantum Entanglement with Classical Communication , 1999 .

[41]  Ronitt Rubinfeld,et al.  Learning Polynomials with Queries: The Highly Noisy Case , 2000, SIAM J. Discret. Math..

[42]  B. Tsirelson.  Quantum analogues of the Bell inequalities. The case of two spatially separated domains , 1987 .

[43]  Cerf,et al.  Classical teleportation of a quantum Bit , 2000, Physical review letters.

[44]  A. Razborov.  Lower bounds on the size of bounded depth circuits over a complete basis with logical addition , 1987 .

[45]  Luca Trevisan,et al.  Some Applications of Coding Theory in Computational Complexity , 2004, Electron. Colloquium Comput. Complex..

[46]  Noga Alon,et al.  Approximating the cut-norm via Grothendieck's inequality , 2004, STOC '04.

[47]  Shachar Lovett,et al.  Unconditional pseudorandom generators for low degree polynomials , 2008, Theory Comput..

[48]  Venkatesan Guruswami,et al.  Improved decoding of Reed-Solomon and algebraic-geometry codes , 1999, IEEE Trans. Inf. Theory.

[49]  Noam Nisan,et al.  On Randomized One-round Communication Complexity , 1995, STOC '95.

[50]  Eyal Kushilevitz,et al.  Learning decision trees using the Fourier spectrum , 1991, STOC '91.

[51]  Venkatesan Guruswami,et al.  List decoding of error correcting codes , 2001 .

[52]  David Zuckerman,et al.  List-decoding reed-muller codes over small fields , 2008, STOC '08.

[53]  Subhash Khot,et al.  Hardness of Minimizing and Learning DNF Expressions , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[54]  Madhu Sudan,et al.  Improved Low-Degree Testing and its Applications , 2003, Comb..

[55]  Yaoyun Shi.  Tensor norms and the classical communication complexity of nonlocal quantum measurement , 2005, STOC '05.