Non-Black-Box Worst-Case to Average-Case Reductions within NP

There are significant obstacles to establishing an equivalence between the worst-case and average-case hardness of NP: Several results suggest that black-box worst-case to average-case reductions are not likely to be used for reducing any worst-case problem outside coNP to a distributional NP problem. This paper overcomes the barrier. We present the first non-black-box worst-case to average-case reduction from a problem outside coNP (unless Random 3SAT is easy for coNP algorithms) to a distributional NP problem. Specifically, we consider the minimum time-bounded Kolmogorov complexity problem (MINKT), and prove that there exists a zero-error randomized polynomial-time algorithm approximating the minimum time-bounded Kolmogorov complexity k within an additive error roughly √k if its average-case version admits an errorless heuristic polynomial-time algorithm. (The converse direction also holds under a plausible derandomization assumption.) We also show that, given a truth table of size 2^n, approximating the minimum circuit size within a factor of 2^(1-e) n is in BPP for some constant e > 0 if and only if its average-case version is easy. Based on our results, we propose a research program for excluding Heuristica, i.e., establishing an equivalence between the worst-case and average-case hardness of NP through the lens of MINKT or the Minimum Circuit Size Problem (MCSP).

[1]  Eric Allender,et al.  New Insights on the (Non-)Hardness of Circuit Minimization and Related Problems , 2019, MFCS.

[2]  Russell Impagliazzo,et al.  One-way functions are essential for complexity based cryptography , 1989, 30th Annual Symposium on Foundations of Computer Science.

[3]  Eric Allender,et al.  Complexity Theory , 1997, Encyclopedia of Cryptography and Security.

[4]  Jin-Yi Cai,et al.  Circuit minimization problem , 2000, STOC '00.

[5]  Igor Carboni Oliveira,et al.  Conspiracies between Learning Algorithms, Circuit Lower Bounds and Pseudorandomness , 2016, CCC.

[6]  Ingo Wegener,et al.  Complexity Theory , 2005 .

[7]  Avi Wigderson,et al.  Algebrization: A New Barrier in Complexity Theory , 2009, TOCT.

[8]  Ker-I Ko,et al.  On the complexity of learning minimum time-bounded Turing machines , 1990, COLT '90.

[9]  S. Vadhan.  Pseudorandomness II , 2009 .

[10]  Luca Trevisan,et al.  On worst-case to average-case reductions for NP problems , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[11]  Avi Wigderson,et al.  Randomness vs Time: Derandomization under a Uniform Assumption , 2001, J. Comput. Syst. Sci..

[12]  Andrew Chi-Chih Yao,et al.  Theory and Applications of Trapdoor Functions (Extended Abstract) , 1982, FOCS.

[13]  Osamu Watanabe,et al.  On Nonadaptive Reductions to the Set of Random Strings and Its Dense Subsets , 2019, Electron. Colloquium Comput. Complex..

[14]  Russell Impagli.  A Personal View of Average-Case Complexity , 1995 .

[15]  John Gill,et al.  Relativizations of the P =? NP Question , 1975, SIAM J. Comput..

[16]  Eric Allender,et al.  Zero knowledge and circuit minimization , 2014, Inf. Comput..

[17]  Noam Nisan,et al.  Hardness vs. randomness , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[18]  Silvio Micali,et al.  How to construct random functions , 1986, JACM.

[19]  Leonid A. Levin,et al.  Average Case Complete Problems , 1986, SIAM J. Comput..

[20]  Dorit Aharonov,et al.  Lattice problems in NP ∩ coNP , 2005, JACM.

[21]  Luca Trevisan,et al.  Pseudorandom generators without the XOR lemma , 1999, Proceedings. Fourteenth Annual IEEE Conference on Computational Complexity (Formerly: Structure in Complexity Theory Conference) (Cat.No.99CB36317).

[22]  Boris A. Trakhtenbrot,et al.  A Survey of Russian Approaches to Perebor (Brute-Force Searches) Algorithms , 1984, Annals of the History of Computing.

[23]  Russell Impagliazzo,et al.  Learning Algorithms from Natural Proofs , 2016, CCC.

[24]  Miklós Ajtai,et al.  Generating hard instances of lattice problems (extended abstract) , 1996, STOC '96.

[25]  Rahul Santhanam,et al.  On the Average-Case Complexity of MCSP and Its Variants , 2017, CCC.

[26]  Luca Trevisan,et al.  Pseudorandomness and average-case complexity via uniform reductions , 2002, Proceedings 17th IEEE Annual Conference on Computational Complexity.

[27]  Igor Carboni Oliveira,et al.  NP-hardness of Minimum Circuit Size Problem for OR-AND-MOD Circuits , 2018, Electron. Colloquium Comput. Complex..

[28]  Amnon Ta-Shma,et al.  If NP Languages are Hard on the Worst-Case, Then it is Easy to Find Their Hard Instances , 2005, 20th Annual IEEE Conference on Computational Complexity (CCC'05).

[29]  Eric Allender,et al.  The pervasive reach of resource-bounded Kolmogorov complexity in computational complexity theory , 2009, J. Comput. Syst. Sci..

[30]  Troy Lee,et al.  Language compression and pseudorandom generators , 2004, Proceedings. 19th IEEE Annual Conference on Computational Complexity, 2004..

[31]  Ilya Volkovich,et al.  The Power of Natural Properties as Oracles , 2018, Electron. Colloquium Comput. Complex..

[32]  Luca Trevisan.  Average-case Complexity , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[33]  Oded Goldreich,et al.  On basing one-way functions on NP-hardness , 2006, STOC '06.

[34]  Michael E. Saks,et al.  Minimizing Disjunctive Normal Form Formulas and AC0 Circuits Given a Truth Table , 2008, SIAM J. Comput..

[35]  Andrej Bogdanov,et al.  On Basing Size-Verifiable One-Way Functions on NP-Hardness , 2015, TCC.

[36]  Eric Allender,et al.  The Minimum Oracle Circuit Size Problem , 2015, STACS.

[37]  Whitfield Diffie,et al.  New Directions in Cryptography , 1976, IEEE Trans. Inf. Theory.

[38]  Joan Feigenbaum,et al.  Random-Self-Reducibility of Complete Sets , 1993, SIAM J. Comput..

[39]  Eric Allender,et al.  Minimum Circuit Size, Graph Isomorphism, and Related Problems , 2017, Electron. Colloquium Comput. Complex..

[40]  Oded Regev,et al.  Tensor-based Hardness of the Shortest Vector Problem to within Almost Polynomial Factors , 2012, Theory Comput..

[41]  Dieter van Melkebeek,et al.  Graph Nonisomorphism Has Subexponential Size Proofs Unless the Polynomial-Time Hierarchy Collapses , 2002, SIAM J. Comput..

[42]  Uriel Feige,et al.  Witnesses for non-satisfiability of dense random 3CNF formulas , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[43]  Steven Rudich,et al.  Super-bits, Demi-bits, and NP/qpoly-natural Proofs , 1997, RANDOM.

[44]  Leonid A. Levin,et al.  A Pseudorandom Generator from any One-way Function , 1999, SIAM J. Comput..

[45]  Oded Goldreich,et al.  On the Limits of Nonapproximability of Lattice Problems , 2000, J. Comput. Syst. Sci..

[46]  Alexander A. Razborov,et al.  Natural Proofs , 1997, J. Comput. Syst. Sci..

[47]  John M. Hitchcock,et al.  On the NP-Completeness of the Minimum Circuit Size Problem , 2015, FSTTCS.

[48]  Uriel Feige,et al.  Easily refutable subformulas of large random 3CNF formulas , 2004, Theory of Computing.

[49]  Leonard M. Adleman,et al.  Two theorems on random polynomial time , 1978, 19th Annual Symposium on Foundations of Computer Science (sfcs 1978).

[50]  Salil P. Vadhan,et al.  An unconditional study of computational zero knowledge , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[51]  Marius Zimand.  A list-decodable code with local encoding and decoding , 2005, Sixth International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing and First ACIS International Workshop on Self-Assembling Wireless Network.

[52]  Emanuele Viola,et al.  The complexity of constructing pseudorandom generators from hard functions , 2005, computational complexity.

[53]  Ran Raz,et al.  Extracting all the randomness and reducing the error in Trevisan's extractors , 1999, STOC '99.

[54]  Luca Trevisan,et al.  Extractors and pseudorandom generators , 2001, JACM.

[55]  Madhu Sudan,et al.  Decoding of Reed Solomon Codes beyond the Error-Correction Bound , 1997, J. Complex..

[56]  Russell Impagliazzo,et al.  Relativized Separations of Worst-Case and Average-Case Complexities for NP , 2011, 2011 IEEE 26th Annual Conference on Computational Complexity.

[57]  Noam Nisan,et al.  BPP has subexponential time simulations unless EXPTIME has publishable proofs , 1991, [1991] Proceedings of the Sixth Annual Structure in Complexity Theory Conference.

[58]  Eric Allender,et al.  Power from random strings , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[59]  Rafail Ostrovsky,et al.  One-way functions, hard on average problems, and statistical zero-knowledge proofs , 1991, [1991] Proceedings of the Sixth Annual Structure in Complexity Theory Conference.

[60]  Dieter van Melkebeek,et al.  Graph nonisomorphism has subexponential size proofs unless the polynomial-time hierarchy collapses , 1999, STOC '99.

[61]  Avi Wigderson,et al.  P = BPP if E requires exponential circuits: derandomizing the XOR lemma , 1997, STOC '97.

[62]  Cody Murray,et al.  On the (Non) NP-Hardness of Computing Circuit Complexity , 2017, Theory Comput..

[63]  Osamu Watanabe,et al.  Limits of Minimum Circuit Size Problem as Oracle , 2016, CCC.

[64]  Uriel Feige,et al.  Resolution lower bounds for the weak pigeon hole principle , 2002, Proceedings 17th IEEE Annual Conference on Computational Complexity.

[65]  Daniele Micciancio,et al.  Worst-case to average-case reductions based on Gaussian measures , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.