On the nature of the Theory of Computation (ToC)

[This paper is a (self contained) chapter in a new book on computational complexity theory, called Mathematics and Computation, available at https://www.math.ias.edu/avi/book]. I attempt to give here a panoramic view of the Theory of Computation, that demonstrates its place as a revolutionary, disruptive science, and as a central, independent intellectual discipline. I discuss many aspects of the field, mainly academic but also cultural and social. The paper details of the rapid expansion of interactions ToC has with all sciences, mathematics and philosophy. I try to articulate how these connections naturally emanate from the intrinsic investigation of the notion of computation itself, and from the methodology of the field. These interactions follow the other, fundamental role that ToC played, and continues to play in the amazing developments of computer technology. I discuss some of the main intellectual goals and challenges of the field, which, together with the ubiquity of computation across human inquiry makes its study and understanding in the future at least as exciting and important

[1]  Ankur Moitra,et al.  Approximate Counting, the Lovász Local Lemma, and Inference in Graphical Models , 2019, J. ACM.

[2]  Noah Fleming.  Approximate Constraint Satisfaction Requires Large LP Relaxations , 2018 .

[3]  Pravesh Kothari,et al.  Outlier-robust moment-estimation via sum-of-squares , 2017, ArXiv.

[4]  Irit Dinur,et al.  High Dimensional Expanders Imply Agreement Expanders , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[5]  Samuel B. Hopkins,et al.  Bayesian estimation from few samples: community detection and related problems , 2017, ArXiv.

[6]  Mark Jerrum,et al.  Uniform sampling through the Lovasz local lemma , 2017, STOC.

[7]  Zeyuan Allen Zhu,et al.  Linear Coupling: An Ultimate Unification of Gradient and Mirror Descent , 2014, ITCS.

[8]  Zahra Derakhshandeh,et al.  Algorithmic Foundations of Self-Organizing Programmable Matter , 2017 .

[9]  Alexander I. Barvinok,et al.  Combinatorics and Complexity of Partition Functions , 2017, Algorithms and combinatorics.

[10]  Liwei Wang,et al.  Stable Memory Allocation in the Hippocampus: Fundamental Limits and Neural Realization , 2016, ArXiv.

[11]  Christos H. Papadimitriou,et al.  Sex as an algorithm , 2016, Commun. ACM.

[12]  Zeev Dvir,et al.  2-Server PIR with Subpolynomial Communication , 2016, J. ACM.

[13]  Scott Aaronson,et al.  The Complexity of Quantum States and Transformations: From Quantum Money to Black Holes , 2016, Electron. Colloquium Comput. Complex..

[14]  Zeyuan Allen Zhu,et al.  Variance Reduction for Faster Non-Convex Optimization , 2016, ICML.

[15]  Yael Tauman Kalai,et al.  Cryptographic Assumptions: A Position Paper , 2016, TCC.

[16]  Avi Wigderson,et al.  A Deterministic Polynomial Time Algorithm for Non-commutative Rational Identity Testing , 2015, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[17]  D. Harlow,et al.  Jerusalem Lectures on Black Holes and Quantum Information , 2014, 1409.1231.

[18]  Christophe Hurlin,et al.  Where the Risks Lie: A Survey on Systemic Risk , 2015 .

[19]  Nir Bitansky,et al.  On the Cryptographic Hardness of Finding a Nash Equilibrium , 2015, FOCS.

[20]  A. Tiwari,et al.  Advanced Theranostic Materials: Tiwari/Advanced , 2015 .

[21]  Santosh S. Vempala,et al.  Cortical Learning via Prediction , 2015, COLT.

[22]  Or Meir,et al.  High-rate locally-correctable and locally-testable codes with sub-polynomial query complexity , 2015, STOC.

[23]  David C. Parkes,et al.  On Sex, Evolution, and the Multiplicative Weights Update Algorithm , 2015, AAMAS.

[24]  B. Chazelle.  An Algorithmic Approach to Collective Behavior , 2015 .

[25]  Prasad Raghavendra,et al.  Lower Bounds on the Size of Semidefinite Programming Relaxations , 2014, STOC.

[26]  Toniann Pitassi,et al.  Preserving Statistical Validity in Adaptive Data Analysis , 2014, STOC.

[27]  Hans Raj Tiwary,et al.  Exponential Lower Bounds for Polytopes in Combinatorial Optimization , 2011, J. ACM.

[28]  Heinz Koeppl,et al.  Self-assembly and Self-organization in Computer Science and Biology (Dagstuhl Seminar 15402) , 2015, Dagstuhl Reports.

[29]  Virginia Vassilevska Williams,et al.  Hardness of Easy Problems: Basing Hardness on Popular Conjectures such as the Strong Exponential Time Hypothesis (Invited Talk) , 2015, IPEC.

[30]  Nir Shavit,et al.  Sparse sign-consistent Johnson–Lindenstrauss matrices: Compression with neuroscience-based constraints , 2014, Proceedings of the National Academy of Sciences.

[31]  Yin Tat Lee,et al.  Path Finding Methods for Linear Programming: Solving Linear Programs in Õ(vrank) Iterations and Faster Algorithms for Maximum Flow , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[32]  Nir Shavit,et al.  The big data challenges of connectomics , 2014, Nature Neuroscience.

[33]  Alex Samorodnitsky,et al.  Bounds on the Permanent and Some Applications , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[34]  Umesh Vazirani,et al.  Algorithms, games, and evolution , 2014, Proceedings of the National Academy of Sciences.

[35]  L. Valiant.  What must a global theory of cortex explain? , 2014, Current Opinion in Neurobiology.

[36]  David Steurer,et al.  Sum-of-squares proofs and the quest toward optimal algorithms , 2014, Electron. Colloquium Comput. Complex..

[37]  Renato Paes Leme,et al.  On the efficiency of the walrasian mechanism , 2013, EC.

[38]  Yin Tat Lee,et al.  An Almost-Linear-Time Algorithm for Approximate Max Flow in Undirected Graphs, and its Multicommodity Generalizations , 2013, SODA.

[39]  Avi Wigderson,et al.  IMPROVED RANK BOUNDS FOR DESIGN MATRICES AND A NEW PROOF OF KELLY’S THEOREM , 2012, Forum of Mathematics, Sigma.

[40]  Aleksander Madry,et al.  Navigating Central Path with Electrical Flows: From Flows to Matchings, and Back , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[41]  L. Valiant.  Probably Approximately Correct: Nature's Algorithms for Learning and Prospering in a Complex World , 2013 .

[42]  Yin Tat Lee,et al.  Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[43]  Maria-Florina Balcan,et al.  Clustering under approximation stability , 2013, JACM.

[44]  P. Hayden,et al.  Quantum computation vs. firewalls , 2013, 1301.4504.

[45]  Oron Shagrir,et al.  Why Philosophers Should Care about Computational Complexity , 2013 .

[46]  László Lovász,et al.  Large Networks and Graph Limits , 2012, Colloquium Publications.

[47]  Bernard Chazelle,et al.  Natural algorithms and influence systems , 2012, CACM.

[48]  Leslie G. Valiant,et al.  The Hippocampus as a Stable Memory Allocator for Cortex , 2012, Neural Computation.

[49]  J. Polchinski,et al.  Black holes: complementarity or firewalls? , 2012, Journal of High Energy Physics.

[50]  Sanjeev Arora,et al.  The Multiplicative Weights Update Method: a Meta-Algorithm and Applications , 2012, Theory Comput..

[51]  Cheng Huang,et al.  On the Locality of Codeword Symbols , 2011, IEEE Transactions on Information Theory.

[52]  Nathan Linial,et al.  Are Stable Instances Easy? , 2009, Combinatorics, Probability and Computing.

[53]  Sanjeev Arora,et al.  Computational complexity and information asymmetry in financial products , 2011, Commun. ACM.

[54]  Shang-Hua Teng,et al.  Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs , 2010, STOC '11.

[55]  C. Mao,et al.  Synergistic self-assembly of RNA and DNA molecules , 2010, Nature chemistry.

[56]  E. David,et al.  Networks, Crowds, and Markets: Reasoning about a Highly Connected World , 2010 .

[57]  Shimon Kogan,et al.  Trading Complex Assets , 2010 .

[58]  Allan Sly,et al.  Computational Transition at the Uniqueness Threshold , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[59]  Adi Livnat,et al.  Sex, mixability, and modularity , 2010, Proceedings of the National Academy of Sciences.

[60]  Gábor Tardos,et al.  A constructive proof of the general lovász local lemma , 2009, JACM.

[61]  Shubhangi Saraf,et al.  Blackbox Polynomial Identity Testing for Depth 3 Circuits , 2009, 2009 50th Annual IEEE Symposium on Foundations of Computer Science.

[62]  Klim Efremenko,et al.  3-Query Locally Decodable Codes of Subexponential Length , 2008 .

[63]  Avi Wigderson,et al.  New direct-product testers and 2-query PCPs , 2009, STOC '09.

[64]  C. Papadimitriou,et al.  The complexity of computing a Nash equilibrium , 2009, CACM.

[65]  Robin A. Moser.  A constructive proof of the Lovász local lemma , 2008, STOC '09.

[66]  Xiaotie Deng,et al.  Settling the complexity of computing two-player Nash equilibria , 2007, JACM.

[67]  Adi Livnat,et al.  A mixability theory for the role of sex in evolution , 2008, Proceedings of the National Academy of Sciences.

[68]  Viola Vogel,et al.  Harnessing biological motors to engineer systems for nanoscale transport and assembly. , 2008, Nature nanotechnology.

[69]  Prasad Raghavendra,et al.  Optimal algorithms and inapproximability results for every CSP? , 2008, STOC.

[70]  Venkatesan Guruswami,et al.  Explicit Codes Achieving List Decoding Capacity: Error-Correction With Optimal Redundancy , 2005, IEEE Transactions on Information Theory.

[71]  Leslie G. Valiant,et al.  A Quantitative Theory of Neural Computation , 2006, Biological Cybernetics.

[72]  Dror Weitz,et al.  Counting independent sets up to the tree threshold , 2006, STOC '06.

[73]  Nicholas Pippenger,et al.  An optimal brain can be composed of conflicting agents. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[74]  H. Markram.  The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[75]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[76]  Erik Winfree,et al.  Self-healing Tile Sets , 2006, Nanotechnology: Science and Computation.

[77]  Erik Winfree,et al.  Two computational primitives for algorithmic self-assembly: copying and counting. , 2005, Nano letters.

[78]  Alexander Vardy,et al.  Correcting errors beyond the Guruswami-Sudan radius in polynomial time , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[79]  Scott Aaronson,et al.  Guest Column: NP-complete problems and physical reality , 2005, SIGA.

[80]  Carsten Lund,et al.  Non-deterministic exponential time has two-prover interactive protocols , 2005, computational complexity.

[81]  E. Winfree,et al.  Algorithmic Self-Assembly of DNA Sierpinski Triangles , 2004, PLoS biology.

[82]  Eric Vigoda,et al.  A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries , 2004, JACM.

[83]  Nadrian C Seeman,et al.  Nanotechnology and the double helix. , 2004, Scientific American.

[84]  E. Shapiro,et al.  An autonomous molecular computer for logical control of gene expression , 2004, Nature.

[85]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[86]  Aranyak Mehta,et al.  Playing large games using simple strategies , 2003, EC '03.

[87]  Oded Goldreich,et al.  Locally testable codes and PCPs of almost-linear length , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[88]  Béla Bollobás,et al.  Proving integrality gaps without knowing the linear program , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[89]  Martin E. Dyer,et al.  Mixing in time and space for lattice spin systems: A combinatorial view , 2002, RANDOM.

[90]  Subhash Khot,et al.  On the power of unique 2-prover 1-round games , 2002, Proceedings 17th IEEE Annual Conference on Computational Complexity.

[91]  Avi Wigderson,et al.  Randomness conductors and constant-degree lossless expanders , 2002, Proceedings 17th IEEE Annual Conference on Computational Complexity.

[92]  Dima Grigoriev,et al.  Complexity of Positivstellensatz proofs for the knapsack , 2002, computational complexity.

[93]  Russell Impagliazzo,et al.  Which Problems Have Strongly Exponential Complexity? , 2001, J. Comput. Syst. Sci..

[94]  Uriel Feige,et al.  Heuristics for Semirandom Graph Problems , 2001, J. Comput. Syst. Sci..

[95]  Boaz Barak,et al.  How to go beyond the black-box simulation barrier , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[96]  David Martin,et al.  Computational Molecular Biology: An Algorithmic Approach , 2001 .

[97]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[98]  Shang-Hua Teng,et al.  Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time , 2001, STOC '01.

[99]  Dima Grigoriev,et al.  Linear lower bound on degrees of Positivstellensatz calculus proofs for the parity , 2001, Theor. Comput. Sci..

[100]  Russell Impagliazzo,et al.  On the Complexity of k-SAT , 2001, J. Comput. Syst. Sci..

[101]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[102]  Daniel A. Spielman,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[103]  Tim Roughgarden,et al.  How bad is selfish routing? , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[104]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[105]  Madhu Sudan.  List decoding: algorithms and applications , 2000, SIGA.

[106]  Christos H. Papadimitriou,et al.  Worst-case Equilibria , 1999, STACS.

[107]  Luca Trevisan,et al.  Pseudorandom generators without the XOR Lemma , 1999, Electron. Colloquium Comput. Complex..

[108]  Venkatesan Guruswami,et al.  Improved decoding of Reed-Solomon and algebraic-geometric codes , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[109]  Alex Samorodnitsky,et al.  A Deterministic Strongly Polynomial Algorithm for Matrix Scaling and Approximate Permanents , 1998, STOC '98.

[110]  J. Maldacena.  The Large-N Limit of Superconformal Field Theories and Supergravity , 1997, hep-th/9711200.

[111]  Oded Goldreich,et al.  A Combinatorial Consistency Lemma with Application to Proving the PCP Theorem , 1997, SIAM J. Comput..

[112]  Madhu Sudan,et al.  Improved Low-Degree Testing and its Applications , 1997, STOC '97.

[113]  Ran Raz,et al.  A sub-constant error-probability low-degree test, and a sub-constant error-probability PCP characterization of NP , 1997, STOC '97.

[114]  Madhu Sudan,et al.  Decoding of Reed Solomon Codes beyond the Error-Correction Bound , 1997, J. Complex..

[115]  László Lovász,et al.  Interactive proofs and the hardness of approximating cliques , 1996, JACM.

[116]  D. Hochbaum.  Approximation Algorithms for NP-Hard Problems , 1996 .

[117]  Daniel A. Spielman,et al.  Linear-time encodable and decodable error-correcting codes , 1995, STOC '95.

[118]  R J Lipton,et al.  DNA solution of hard computational problems. , 1995, Science.

[119]  Leslie G. Valiant,et al.  Circuits of the mind , 1994 .

[120]  L M Adleman,et al.  Molecular computation of solutions to combinatorial problems. , 1994, Science.

[121]  Christos H. Papadimitriou,et al.  On the Complexity of the Parity Argument and Other Inefficient Proofs of Existence , 1994, J. Comput. Syst. Sci..

[122]  Mark Jerrum,et al.  Simulated annealing for graph bisection , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[123]  Leonard J. Schulman,et al.  Deterministic coding for interactive communication , 1993, STOC.

[124]  Sanjeev Arora,et al.  Probabilistic checking of proofs; a new characterization of NP , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[125]  Leonard J. Schulman,et al.  Communication on noisy channels: a coding theorem for computation , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[126]  Carsten Lund,et al.  Proof verification and hardness of approximation problems , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[127]  Mark Jerrum,et al.  Large Cliques Elude the Metropolis Process , 1992, Random Struct. Algorithms.

[128]  József Beck,et al.  An Algorithmic Approach to the Lovász Local Lemma. I , 1991, Random Struct. Algorithms.

[129]  Silvio Micali,et al.  Proofs that yield nothing but their validity or all languages in NP have zero-knowledge proof systems , 1991, JACM.

[130]  Manuel Blum,et al.  Self-testing/correcting with applications to numerical problems , 1990, STOC '90.

[131]  Michael R. Fellows,et al.  Computer science and mathematics in the elementary schools , 1990 .

[132]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[133]  Mark Jerrum,et al.  Approximating the Permanent , 1989, SIAM J. Comput..

[134]  Leonid A. Levin,et al.  A hard-core predicate for all one-way functions , 1989, STOC '89.

[135]  Jean,et al.  The Computer and the Brain , 1989, Annals of the History of Computing.

[136]  Mihalis Yannakakis,et al.  Expressing combinatorial optimization problems by linear programs , 1991, STOC '88.

[137]  Avi Wigderson,et al.  Completeness theorems for non-cryptographic fault-tolerant distributed computation , 1988, STOC '88.

[138]  Mark Jerrum,et al.  Approximate Counting, Uniform Generation and Rapidly Mixing Markov Chains , 1987, WG.

[139]  Silvio Micali,et al.  How to play ANY mental game , 1987, STOC.

[140]  David A. Mix Barrington,et al.  Bounded-width polynomial-size branching programs recognize exactly those languages in NC1 , 1986, STOC '86.

[141]  Leslie G. Valiant,et al.  Random Generation of Combinatorial Structures from a Uniform Distribution , 1986, Theor. Comput. Sci..

[142]  Leonid A. Levin,et al.  Average Case Complete Problems , 1986, SIAM J. Comput..

[143]  A. Yao.  How to generate and exchange secrets , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[144]  D. Deutsch.  Quantum theory, the Church–Turing principle and the universal quantum computer , 1985, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[145]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[146]  R. Feynman.  Simulating physics with computers , 1999 .

[147]  P. Benioff.  The computer as a physical system: A microscopic quantum mechanical Hamiltonian model of computers as represented by Turing machines , 1980 .

[148]  Leslie G. Valiant,et al.  The Complexity of Enumeration and Reliability Problems , 1979, SIAM J. Comput..

[149]  Leslie G. Valiant,et al.  The Complexity of Computing the Permanent , 1979, Theor. Comput. Sci..

[150]  J. Schwartz,et al.  Theory of Self-Reproducing Automata , 1967 .

[151]  Sharon L. Milgram,et al.  The Small World Problem , 1967 .

[152]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[153]  E. Wigner.  The Unreasonable Effectiveness of Mathematics in the Natural Sciences (reprint) , 1960 .

[154]  Peter Elias,et al.  List decoding for noisy channels , 1957 .

[155]  A. M. Turing,et al.  The chemical basis of morphogenesis , 1952, Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences.