Bias vs structure of polynomials in large fields, and applications in effective algebraic geometry and coding theory

Let $f$ be a polynomial of degree $d$ in $n$ variables over a finite field $\mathbb{F}$. The polynomial is said to be unbiased if the distribution of $f(x)$ for a uniform input $x \in \mathbb{F}^n$ is close to the uniform distribution over $\mathbb{F}$, and is called biased otherwise. The polynomial is said to have low rank if it can be expressed as a composition of a few lower degree polynomials. Green and Tao [Contrib. Discrete Math 2009] and Kaufman and Lovett [FOCS 2008] showed that bias implies low rank for fixed degree polynomials over fixed prime fields. This lies at the heart of many tools in higher order Fourier analysis. In this work, we extend this result to all prime fields (of size possibly growing with $n$). We also provide a generalization to nonprime fields in the large characteristic case. However, we state all our applications in the prime field setting for the sake of simplicity of presentation. As an immediate application, we obtain improved bounds for a suite of problems in effective algebraic geometry, including Hilbert nullstellensatz, radical membership and counting rational points in low degree varieties. Using the above generalization to large fields as a starting point, we are also able to settle the list decoding radius of fixed degree Reed-Muller codes over growing fields. The case of fixed size fields was solved by Bhowmick and Lovett [STOC 2015], which resolved a conjecture of Gopalan-Klivans-Zuckerman [STOC 2008]. Here, we show that the list decoding radius is equal the minimum distance of the code for all fixed degrees, even when the field size is possibly growing with $n$.

[1]  Ming-Deh A. Huang,et al.  An Algorithm for Approximate Counting of Points on Algebraic Sets over Finite Fields , 1998, ANTS.

[2]  Ben Green,et al.  The distribution of polynomials over finite fields, with applications to the Gowers norms , 2007, Contributions Discret. Math..

[3]  Peter Elias,et al.  List decoding for noisy channels , 1957 .

[4]  Eyal Kushilevitz,et al.  Learning decision trees using the Fourier spectrum , 1991, STOC '91.

[5]  Grete Hermann,et al.  Die Frage der endlich vielen Schritte in der Theorie der Polynomideale , 1926 .

[6]  Leonid A. Levin,et al.  A hard-core predicate for all one-way functions , 1989, STOC '89.

[7]  Andrej Bogdanov Pseudorandom generators for low degree polynomials , 2005, STOC '05.

[8]  Christopher Umans,et al.  Simple extractors for all min-entropies and a new pseudorandom generator , 2005, JACM.

[9]  Leonard M. Adleman,et al.  Counting Rational Points on Curves and Abelian Varieties over Finite Fields , 1996, ANTS.

[10]  Luca Trevisan,et al.  List-decoding using the XOR lemma , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[11]  Abhishek Bhowmick,et al.  Using higher-order Fourier analysis over general fields , 2015, Electron. Colloquium Comput. Complex..

[12]  Parikshit Gopalan,et al.  A Fourier-Analytic Approach to Reed–Muller Decoding , 2010, IEEE Transactions on Information Theory.

[13]  Selmer M. Johnson A new upper bound for error-correcting codes , 1962, IRE Trans. Inf. Theory.

[14]  Madhu Sudan,et al.  Decodability of group homomorphisms beyond the johnson bound , 2008, STOC '08.

[15]  Ronitt Rubinfeld,et al.  Learning polynomials with queries: The highly noisy case , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[16]  Jeffrey C. Jackson,et al.  An efficient membership-query algorithm for learning DNF with respect to the uniform distribution , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[17]  Emanuele Viola The Sum of d Small-Bias Generators Fools Polynomials of Degree d , 2008, Computational Complexity Conference.

[18]  Ming-Deh A. Huang,et al.  Counting rational points on curves over finite fields , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[19]  Sergey Yekhanin,et al.  Detecting Rational Points on Hypersurfaces over Finite Fields , 2008, 2008 23rd Annual IEEE Conference on Computational Complexity.

[20]  Arnab Bhattacharyya Polynomial Decompositions in Polynomial Time , 2014, ESA.

[21]  Parikshit Gopalan A Fourier-Analytic Approach to Reed-Muller Decoding , 2013, IEEE Trans. Inf. Theory.

[22]  Madhur Tulsiani,et al.  Algorithmic regularity for polynomials and applications , 2013, SODA.

[23]  Emanuele Viola,et al.  Pseudorandom Bits for Polynomials , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[24]  Venkatesan Guruswami,et al.  Algorithms for Modular Counting of Roots of Multivariate Polynomials , 2006, Algorithmica.

[25]  Marek Karpinski,et al.  Counting curves and their projections , 1993, computational complexity.

[26]  E. Szemerédi Regular Partitions of Graphs , 1975 .

[27]  W. Brownawell Bounds for the degrees in the Nullstellensatz , 1987 .

[28]  Shachar Lovett Holes in Generalized Reed–Muller Codes , 2010, IEEE Transactions on Information Theory.

[29]  Vladimir M. Blinovsky,et al.  List decoding , 1992, Discret. Math..

[30]  Amnon Ta-Shma,et al.  Extractors from Reed-Muller Codes , 2001, Electron. Colloquium Comput. Complex..

[31]  Shachar Lovett,et al.  Every locally characterized affine-invariant property is testable , 2013, STOC '13.

[32]  Shulim Kaliman,et al.  On a theorem of Ax , 2004 .

[33]  J. Kollár Sharp effective Nullstellensatz , 1988 .

[34]  Shafi Goldwasser,et al.  Proving hard-core predicates using list decoding , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[35]  Luca Trevisan,et al.  Pseudorandom generators without the XOR lemma , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[36]  Enkatesan G Uruswami Unbalanced expanders and randomness extractors from Parvaresh-Vardy codes , 2008 .

[37]  Terence Tao,et al.  Expanding polynomials over finite fields of large characteristic, and a regularity lemma for definable sets , 2012, Contributions Discret. Math..

[38]  Alan M. Frieze,et al.  Quick Approximation to Matrices and Applications , 1999, Comb..

[39]  Shachar Lovett,et al.  Worst Case to Average Case Reductions for Polynomials , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.