SIGACT News Complexity Theory Column 76: an atypical survey of typical-case heuristic algorithms

Heuristic approaches often do so well that they seem to pretty much always give the right answer. How close can heuristic algorithms get to always giving the right answer, without inducing seismic complexity-theoretic consequences? This article first discusses how a series of results by Berman, Buhrman, Hartmanis, Homer, Longpré, Ogiwara, Schöning, and Watanabe, from the early 1970s through the early 1990s, explicitly or implicitly limited how well heuristic algorithms can do on NP-hard problems. In particular, many desirable levels of heuristic success cannot be obtained unless severe, highly unlikely complexity class collapses occur. Second, we survey work initiated by Goldreich and Wigderson, who showed how under plausible assumptions deterministic heuristics for randomized computation can achieve a very high frequency of correctness. Finally, we consider formal ways in which theory can help explain the effectiveness of heuristics that solve NP-hard problems in practice.

[1]  Harry Buhrman,et al.  NP-Hard Sets Are Exponentially Dense Unless coNP C NP/poly , 2008, 2008 23rd Annual IEEE Conference on Computational Complexity.

[2]  Juris Hartmanis,et al.  On isomorphisms and density of NP and other complete sets , 1976, STOC '76.

[3]  Armin Haken,et al.  The Intractability of Resolution , 1985, Theor. Comput. Sci..

[4]  Stephen R. Mahaney Sparse complete sets for NP: Solution of a conjecture of Berman and Hartmanis , 1980, 21st Annual Symposium on Foundations of Computer Science (sfcs 1980).

[5]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[6]  Juris Hartmanis,et al.  An Eassay about Research on Sparse NP Complete Sets , 1980, MFCS.

[7]  John T. Gill,et al.  Computational complexity of probabilistic Turing machines , 1974, STOC '74.

[8]  Paul Young How reductions to sparse sets collapse the polynomial-time hierarchy: a primer: Part II restricted polynomial-time reductions , 1992, SIGA.

[9]  Nancy A. Lynch,et al.  Comparison of polynomial-time reducibilities , 1974, STOC '74.

[10]  Paul Young How reductions to sparse sets collapse the polynomial-time hierarchy: a primer; part I: polynomial-time Turing reductions , 1992, SIGA.

[11]  Harry Buhrman,et al.  Superpolynomial Circuits, Almost Sparse Oracles and the Exponential Hierarchy , 1992, FSTTCS.

[12]  Ariel D. Procaccia,et al.  Junta distributions and the average-case complexity of manipulating elections , 2006, AAMAS '06.

[13]  Richard J. Lipton,et al.  Some connections between nonuniform and uniform complexity classes , 1980, STOC '80.

[14]  Boaz Barak Truth vs. Proof in Computational Complexity , 2012, Bull. EATCS.

[15]  Bart Selman,et al.  Satisfiability Solvers , 2008, Handbook of Knowledge Representation.

[16]  Uwe Schöning,et al.  Complete sets and closeness to complexity classes , 1986, Mathematical systems theory.

[17]  Omer Reingold,et al.  Undirected connectivity in log-space , 2008, JACM.

[18]  Shmuel Safra,et al.  Hardness Amplification for Errorless Heuristics , 2007, 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07).

[19]  Luca Trevisan Average-case Complexity , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[20]  Osamu Watanabe,et al.  How hard are sparse sets? , 1992, [1992] Proceedings of the Seventh Annual Structure in Complexity Theory Conference.

[21]  S. Homer,et al.  On reductions of NP sets to sparse sets , 1991, [1991] Proceedings of the Sixth Annual Structure in Complexity Theory Conference.

[22]  Osamu Watanabe,et al.  On polynomial time bounded truth-table reducibility of NP sets to sparse sets , 1990, STOC '90.

[23]  Russell Impagliazzo,et al.  A personal view of average-case complexity , 1995, Proceedings of Structure in Complexity Theory. Tenth Annual IEEE Conference.

[24]  Marko Samer,et al.  Backdoor Trees , 2008, AAAI.

[25]  Jörg Rothe,et al.  Generalized juntas and NP-hard sets , 2009, Theor. Comput. Sci..

[26]  Juris Hartmanis,et al.  On Isomorphisms and Density of NP and Other Complete Sets , 1977, SIAM J. Comput..

[27]  Bart Selman,et al.  Backdoors To Typical Case Complexity , 2003, IJCAI.

[28]  Ashish Sabharwal,et al.  Backdoors in the Context of Learning , 2009, SAT.

[29]  Rahul Santhanam,et al.  Holographic Proofs and Derandmization , 2005, SIAM J. Comput..

[30]  Russell Impagliazzo,et al.  Relativized Separations of Worst-Case and Average-Case Complexities for NP , 2011, 2011 IEEE 26th Annual Conference on Computational Complexity.

[31]  Eli Ben-Sasson,et al.  Linear Upper Bounds for Random Walk on Small Density Random 3-CNFs , 2007, SIAM J. Comput..

[32]  Christian Glaßer,et al.  A moment of perfect clarity II: consequences of sparse sets hard for NP with respect to weak reductions , 2000, SIGA.

[33]  Eli Ben-Sasson,et al.  Short proofs are narrow—resolution made simple , 2001, JACM.

[34]  Uriel Feige,et al.  Relations between average case complexity and approximation complexity , 2002, STOC '02.

[35]  Michael E. Saks,et al.  The Efficiency of Resolution and Davis--Putnam Procedures , 2002, SIAM J. Comput..

[36]  Avi Wigderson,et al.  Derandomization that is rarely wrong from short advice that is typically good , 2002, Electron. Colloquium Comput. Complex..

[37]  Rahul Santhanam,et al.  Holographic Proofs and Derandomization , 2003, Computational Complexity Conference.

[38]  Neil Immerman,et al.  Sparse sets in NP-P: Exptime versus nexptime , 1983, STOC.

[39]  Marius Zimand Exposure-Resilient Extractors and the Derandomization of Probabilistic Sublinear Time , 2008, computational complexity.

[40]  Alexander A. Razborov,et al.  Complexity of Propositional Proofs , 2010, CSR.

[41]  Robert J. Vanderbei,et al.  Linear Programming: Foundations and Extensions , 1998, Kluwer international series in operations research and management service.

[42]  Salil P. Vadhan,et al.  Computational Complexity , 2005, Encyclopedia of Cryptography and Security.

[43]  Michael Sipser,et al.  Parity, circuits, and the polynomial-time hierarchy , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[44]  Ashish Sabharwal,et al.  Tradeoffs in the Complexity of Backdoor Detection , 2007, CP.

[45]  Avi Wigderson,et al.  Towards a Study of Low-Complexity Graphs , 2009, ICALP.

[46]  Russell Impagliazzo,et al.  Derandomizing Polynomial Identity Tests Means Proving Circuit Lower Bounds , 2003, STOC '03.

[47]  Bart Selman,et al.  Boosting Combinatorial Search Through Randomization , 1998, AAAI/IAAI.

[48]  Anna Slobodová,et al.  Replacing Testing with Formal Verification in Intel CoreTM i7 Processor Execution Engine Validation , 2009, CAV.

[49]  Piotr Faliszewski,et al.  The complexity of manipulative attacks in nearly single-peaked electorates , 2011, TARK XIII.

[50]  Leonid A. Levin,et al.  Average Case Complete Problems , 1986, SIAM J. Comput..

[51]  J. Cai,et al.  S^p _2 \subseteq ZPP^{NP} , 2001, FOCS 2001.

[52]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[53]  Yaacov Yesha,et al.  On Certain Polynomial-Time Truth-Table Reducibilities of Complete Sets to Sparse Sets , 1983, SIAM J. Comput..

[54]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[55]  Neil Immerman,et al.  Sparse Sets in NP-P: EXPTIME versus NEXPTIME , 1985, Inf. Control..

[56]  Jin-Yi Cai,et al.  Competing provers yield improved Karp-Lipton collapse results , 2005, Inf. Comput..

[57]  Dieter van Melkebeek,et al.  Pseudorandom Generators, Typically-Correct Derandomization, and Circuit Lower Bounds , 2011, computational complexity.

[58]  Stefan Szeider,et al.  Backdoors to Satisfaction , 2011, The Multivariate Algorithmic Revolution and Beyond.

[59]  Luca Trevisan,et al.  Lecture Notes on Computational Complexity , 2004 .

[60]  Ronen Shaltiel,et al.  Typically-correct derandomization , 2010, SIGA.

[61]  Yuri Malitsky,et al.  Backdoors to Combinatorial Optimization: Feasibility and Optimality , 2009, CPAIOR.