Derandomizing Polynomial Identity Tests Means Proving Circuit Lower Bounds

Abstract.We show that derandomizing Polynomial Identity Testing is essentially equivalent to proving arithmetic circuit lower bounds for NEXP. More precisely, we prove that if one can test in polynomial time (or even nondeterministic subexponential time, infinitely often) whether a given arithmetic circuit over integers computes an identically zero polynomial, then either (i) % MathType!Translator!2!1!AMS LaTeX.tdl!TeX -- AMS-LaTeX! % MathType!MTEF!2!1!+- % feaafiart1ev1aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr % 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9 % vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x % fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaGaaeOtaiaabw % eacaqGybGaaeiuaiabgsOillaabcfaruqtLrxyqXwDZj0BSrwldfgC % ZbacfaGaa83laiaabchacaqGVbGaaeiBaiaabMhacaqGGaGaae4Bai % aabkhaaaa!4992! $${\text{NEXP}} \not\subset {\text{P}}/{\text{poly or}}$$ (ii) Permanent is not computable by polynomial-size arithmetic circuits. We also prove a (partial) converse: If Permanent requires superpolynomial-size arithmetic circuits, then one can test in subexponential time whether a given arithmetic circuit of polynomially bounded degree computes an identically zero polynomial.Since Polynomial Identity Testing is a coRP problem, we obtain the following corollary: If % MathType!Translator!2!1!AMS LaTeX.tdl!TeX -- AMS-LaTeX! % MathType!MTEF!2!1!+- % feaafiart1ev1aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr % 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9 % vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x % fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaGaaeOuaiaabc % facaqGGaGaaeypaiaabccacaqGqbGaaGjcVlaabIcacaqGVbGaaeOC % aiaabccacaqGLbGaaeODaiaabwgacaqGUbGaaeiiaiaabogacaqGVb % GaaeOuaiaabcfacaqGGaGaeyOHI0SaeSykIK0aaSbaaSqaaiabew7a % Ljabg6da+iaaicdaaeqaaOGaaeiiaiaab6eacaqGubGaaeysaiaab2 % eacaqGfbGaaiikaiaaikdadaahaaWcbeqaaiaad6gadaahaaadbeqa % aiabew7aLbaaaaGccaGGPaGaaiilaiaabccacaqGPbGaaeOBaiaabA % gacaqGPbGaaeOBaiaabMgacaqG0bGaaeyzaiaabYgacaqG5bGaaeii % aiaab+gacaqGMbGaaeiDaiaabwgacaqGUbGaaeykaiaabYcaaaa!6992! $${\text{RP = P}}{\kern 1pt} {\text{(or even coRP }} \subseteq \cap _{\varepsilon > 0} {\text{ NTIME}}(2^{n^\varepsilon } ),{\text{ infinitely often),}}$$ then NEXP is not computable by polynomial-size arithmetic circuits. Thus establishing that RP = coRP or BPP = P would require proving superpolynomial lower bounds for Boolean or arithmetic circuits. We also show that any derandomization of RNC would yield new circuit lower bounds for a language in NEXP.We also prove unconditionally that NEXPRP does not have polynomial-size Boolean or arithmetic circuits. Finally, we show that % MathType!Translator!2!1!AMS LaTeX.tdl!TeX -- AMS-LaTeX! % MathType!MTEF!2!1!+- % feaafiart1ev1aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn % hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr % 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq-Jc9 % vqaqpepm0xbba9pwe9Q8fs0-yqaqpepae9pg0FirpepeKkFr0xfr-x % fr-xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaGaaeOtaiaabw % eacaqGybGaaeiuaiabgsOillaabcfacaqGVaGaaeiCaiaab+gacaqG % SbGaaeyEaaaa!4087! $${\text{NEXP}} \not\subset {\text{P/poly}}$$ if both BPP = P and low-degree testing is in P; here low-degree testing is the problem of checking whether a given Boolean circuit computes a function that is close to some low-degree polynomial over a finite field.

[1]  Avi Wigderson,et al.  Randomness vs. time: de-randomization under a uniform assumption , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[2]  Rafael Hirschfeld,et al.  Pseudorandom Generators and Complexity Classes , 1989, Advances in Computational Research.

[3]  Marek Karpinski,et al.  On Zero-Testing and Interpolation of k-Sparse Multivariate Polynomials Over Finite Fields , 1991, Theor. Comput. Sci..

[4]  Zhi-Zhong Chen,et al.  Reducing randomness via irrational numbers , 1997, STOC '97.

[5]  José D. P. Rolim,et al.  A new general derandomization method , 1998, JACM.

[6]  K. Ramachandra,et al.  Vermeidung von Divisionen. , 1973 .

[7]  Michael E. Saks,et al.  Exponential lower bounds for depth 3 Boolean circuits , 1997, STOC '97.

[8]  Stephen A. Cook,et al.  Efficiently Approximable Real-Valued Functions , 2000, Electron. Colloquium Comput. Complex..

[9]  Erich Kaltofen,et al.  Greatest common divisors of polynomials given by straight-line programs , 1988, JACM.

[10]  Marek Karpinski,et al.  Fast Parallel Algorithms for Sparse Multivariate Polynomial Interpolation over Finite Fields , 1988, SIAM J. Comput..

[11]  László Lovász,et al.  Interactive proofs and the hardness of approximating cliques , 1996, JACM.

[12]  Journal of the Association for Computing Machinery , 1961, Nature.

[13]  Ran Raz,et al.  A sub-constant error-probability low-degree test, and a sub-constant error-probability PCP characterization of NP , 1997, STOC '97.

[14]  Manuel Blum,et al.  Equivalence of Free Boolean Graphs can be Decided Probabilistically in Polynomial Time , 1980, Inf. Process. Lett..

[15]  Manuel Blum,et al.  Designing programs that check their work , 1989, STOC '89.

[16]  Lance Fortnow,et al.  One-sided Versus Two-sided Error in Probabilistic Computation , 1999, STACS.

[17]  Leslie G. Valiant,et al.  The Complexity of Computing the Permanent , 1979, Theor. Comput. Sci..

[18]  Lance Fortnow,et al.  Nonrelativizing separations , 1998, Proceedings. Thirteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat. No.98CB36247).

[19]  Carsten Lund,et al.  Algebraic methods for interactive proof systems , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[20]  Manindra Agrawal,et al.  PRIMES is in P , 2004 .

[21]  Ronitt Rubinfeld,et al.  Robust Characterizations of Polynomials with Applications to Program Testing , 1996, SIAM J. Comput..

[22]  Richard P. Brent,et al.  The Parallel Evaluation of General Arithmetic Expressions , 1974, JACM.

[23]  Richard J. Lipton,et al.  A Probabilistic Remark on Algebraic Program Testing , 1978, Inf. Process. Lett..

[24]  Avi Wigderson,et al.  In search of an easy witness: exponential time vs. probabilistic polynomial time , 2001, Proceedings 16th Annual IEEE Conference on Computational Complexity.

[25]  Erich Kaltofen,et al.  On computing determinants of matrices without divisions , 1992, ISSAC '92.

[26]  Richard J. Lipton,et al.  New Directions In Testing , 1989, Distributed Computing And Cryptography.

[27]  Madhu Sudan,et al.  Highly Resilient Correctors for Polynomials , 1992, Inf. Process. Lett..

[28]  Leslie G. Valiant,et al.  Completeness classes in algebra , 1979, STOC.

[29]  Vaughan R. Pratt,et al.  Every Prime has a Succinct Certificate , 1975, SIAM J. Comput..

[30]  Madhu Sudan,et al.  Decoding of Reed Solomon Codes beyond the Error-Correction Bound , 1997, J. Complex..

[31]  Alexander A. Razborov,et al.  Natural Proofs , 2007 .

[32]  László Babai,et al.  Trading group theory for randomness , 1985, STOC '85.

[33]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1989, 30th Annual Symposium on Foundations of Computer Science.

[34]  Noam Nisan,et al.  BPP has subexponential time simulations unless EXPTIME has publishable proofs , 1991, [1991] Proceedings of the Sixth Annual Structure in Complexity Theory Conference.

[35]  László Lovász,et al.  On determinants, matchings, and random algorithms , 1979, FCT.

[36]  Manindra Agrawal,et al.  Primality and identity testing via Chinese remaindering , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[37]  KaltofenErich Greatest common divisors of polynomials given by straight-line programs , 1988 .

[38]  Avi Wigderson,et al.  Extractors and pseudo-random generators with optimal seed length , 2000, STOC '00.

[39]  Ran Raz,et al.  Extracting all the randomness and reducing the error in Trevisan's extractors , 1999, STOC '99.

[40]  Samuel R. Buss,et al.  An Optimal Parallel Algorithm for Formula Evaluation , 1992, SIAM J. Comput..

[41]  Richard Zippel,et al.  Probabilistic algorithms for sparse polynomials , 1979, EUROSAM.

[42]  Avi Wigderson,et al.  P = BPP if E requires exponential circuits: derandomizing the XOR lemma , 1997, STOC '97.

[43]  Vijay V. Vazirani,et al.  Matching is as easy as matrix inversion , 1987, STOC.

[44]  Michael E. Saks,et al.  An improved exponential-time algorithm for k-SAT , 2005, JACM.

[45]  Daniel A. Spielman,et al.  Randomness efficient identity testing of multivariate polynomials , 2001, STOC '01.

[46]  Jacob T. Schwartz,et al.  Fast Probabilistic Algorithms for Verification of Polynomial Identities , 1980, J. ACM.

[47]  Joachim von zur Gathen,et al.  Feasible Arithmetic Computations: Valiant's Hypothesis , 1987, J. Symb. Comput..

[48]  Salil P. Vadhan,et al.  Checking polynomial identities over any field: towards a derandomization? , 1998, STOC '98.

[49]  Christopher Umans Pseudo-random generators for all hardnesses , 2002, STOC '02.

[50]  Noam Nisan,et al.  Hardness vs Randomness , 1994, J. Comput. Syst. Sci..

[51]  Carsten Lund,et al.  Proof verification and hardness of approximation problems , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[52]  Joan Feigenbaum,et al.  Hiding Instances in Multioracle Queries , 1990, STACS.

[53]  Ron M. Roth,et al.  Interpolation and Approximation of Sparse Multivariate Polynomials over GF(2) , 1991, SIAM J. Comput..

[54]  Adi Shamir,et al.  IP = PSPACE , 1992, JACM.

[55]  Oded Goldreich,et al.  Another proof that bpp?ph (and more) , 1997 .

[56]  Seinosuke Toda,et al.  PP is as Hard as the Polynomial-Time Hierarchy , 1991, SIAM J. Comput..

[57]  Avi Wigderson,et al.  Near-optimal conversion of hardness into pseudo-randomness , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[58]  PaturiRamamohan,et al.  An improved exponential-time algorithm for k-SAT , 2005 .

[59]  Luca Trevisan,et al.  Pseudorandom generators without the XOR lemma , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[60]  Aravind Srinivasan,et al.  Randomness-optimal unique element isolation, with applications to perfect matching and related problems , 1993, SIAM J. Comput..

[61]  Avi Wigderson,et al.  Randomness vs Time: Derandomization under a Uniform Assumption , 2001, J. Comput. Syst. Sci..

[62]  Andrew Chi-Chih Yao,et al.  Theory and Applications of Trapdoor Functions (Extended Abstract) , 1982, FOCS.

[63]  Aravind Srinivasan,et al.  Randomness-Optimal Unique Element Isolation with Applications to Perfect Matching and Related Problems , 1995, SIAM J. Comput..

[64]  Carsten Lund,et al.  Non-deterministic exponential time has two-prover interactive protocols , 2005, computational complexity.

[65]  Erich Kaltofen,et al.  Factorization of Polynomials Given by Straight-Line Programs , 1989, Adv. Comput. Res..

[66]  Madhu Sudan,et al.  Improved Low-Degree Testing and its Applications , 1997, STOC '97.

[67]  Ramamohan Paturi,et al.  Circuits, cnfs, and satisfiability , 1998 .

[68]  Sanjeev Arora,et al.  Probabilistic checking of proofs: a new characterization of NP , 1998, JACM.

[69]  Leslie G. Valiant,et al.  NP is as easy as detecting unique solutions , 1985, STOC '85.

[70]  Valentine Kabanets,et al.  Easiness assumptions and hardness tests: trading time for zero error , 2000, Proceedings 15th Annual IEEE Conference on Computational Complexity.

[71]  Dieter van Melkebeek,et al.  Graph Nonisomorphism Has Subexponential Size Proofs Unless the Polynomial-Time Hierarchy Collapses , 2002, SIAM J. Comput..

[72]  Avi Wigderson,et al.  In search of an easy witness: exponential time vs. probabilistic polynomial time , 2002, J. Comput. Syst. Sci..

[73]  László Babai,et al.  Arthur-Merlin Games: A Randomized Proof System, and a Hierarchy of Complexity Classes , 1988, J. Comput. Syst. Sci..

[74]  Alexander L. Chistov,et al.  Fast parallel calculation of the rank of matrices over a field of arbitrary characteristic , 1985, FCT.

[75]  Michael E. Saks,et al.  Exponential lower bounds for depth three Boolean circuits , 2000, computational complexity.

[76]  Pavel Pudlák,et al.  Satisfiability Coding Lemma , 1997, Proceedings 38th Annual Symposium on Foundations of Computer Science.

[77]  Christopher Umans,et al.  Simple extractors for all min-entropies and a new pseudo-random generator , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[78]  Oscar H. Ibarra,et al.  Probabilistic Algorithms for Deciding Equivalence of Straight-Line Programs , 1983, JACM.