Beyond Number of Bit Erasures: New Complexity Questions Raisedby Recently discovered thermodynamic costs of computation

Recent advances in nonequilibrium statistical mechanics have led to a deeper understanding of the thermodynamic cost of computation than that put forth by Landauer and then studied extensively in the computational complexity community. In particular, Landauer's work led to a focus on the number of bit erasures in a computation, due to its relation to the change in entropy between input and output. However new advances in physics|which have been experimentally con rmed|mean we can now calculate additional thermodynamic costs beyond merely the change in entropy between input and output. As a consequence, we now understand that while logically reversible computing can have some thermodynamic bene ts, it is far from the end of the story. The purpose of this paper is to highlight new open questions in computational complexity raised by consideration of these new thermodynamic costs. Beyond leading to a revised viewpoint on the bene ts of logical reversibility, these questions touch on randomized algorithms, average-case complexity, the thermodynamic cost of error correcting codes, and noisy/inexact/approximate computation.

[1]  Linyuan Lu,et al.  Complex Graphs and Networks (CBMS Regional Conference Series in Mathematics) , 2006 .

[2]  Klaus Sutner,et al.  The complexity of reversible cellular automata , 2004, Theor. Comput. Sci..

[3]  Seth Lloyd,et al.  Information-theoretic approach to the study of control systems , 2001, physics/0104007.

[4]  Richard Ryan Williams,et al.  Probabilistic rank and matrix rigidity , 2016, STOC.

[5]  H. Geiringer.  On the Foundations of Probability Theory , 1967 .

[6]  Dean J. Driebe,et al.  Generalization of the second law for a transition between nonequilibrium states , 2010 .

[7]  David H. Wolpert,et al.  A space–time tradeoff for implementing a function with master equation dynamics , 2017, Nature Communications.

[8]  Blake S. Pollard.  A Second Law for Open Markov Processes , 2016, Open Syst. Inf. Dyn..

[9]  Tim Palmer,et al.  Stochastic modelling and energy-efficient computing for weather and climate prediction , 2014, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[10]  Egon Balas,et al.  A lift-and-project cutting plane algorithm for mixed 0–1 programs , 1993, Math. Program..

[11]  Leonid A. Levin,et al.  Average Case Complete Problems , 1986, SIAM J. Comput..

[12]  Mikhail Prokopenko,et al.  Information thermodynamics of near-equilibrium computation. , 2015, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  R. Landauer.  The physical nature of information , 1996 .

[14]  J. Neumann.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[15]  James P. Crutchfield,et al.  Above and Beyond the Landauer Bound: Thermodynamics of Modularity , 2017, ArXiv.

[16]  Duncan J. Watts,et al.  The Structure and Dynamics of Networks: (Princeton Studies in Complexity) , 2006 .

[17]  L. Brillouin,et al.  The Negentropy Principle of Information , 1953 .

[18]  Charles H. Bennett,et al.  The thermodynamics of computation—a review , 1982 .

[19]  M. B. Plenio,et al.  The physics of forgetting: Landauer's erasure principle and information theory , 2001, quant-ph/0103108.

[20]  Pierre McKenzie,et al.  Reversible space equals deterministic space , 1997, Proceedings of Computational Complexity. Twelfth Annual IEEE Conference.

[21]  Nicole Yunger Halpern,et al.  Introducing one-shot work into fluctuation relations , 2014, 1409.3878.

[22]  John Augustine,et al.  Sustaining Moore's Law Through Inexactness , 2017, ArXiv.

[23]  Harry Buhrman,et al.  Time and Space Bounds for Reversible Simulation , 2001, ICALP.

[24]  Erik D. Demaine,et al.  Toward an Energy Efficient Language and Compiler for (Partially) Reversible Algorithms , 2016, RC.

[25]  Hal Finkel,et al.  Doing Moore with Less - Leapfrogging Moore's Law with Inexactness for Supercomputing , 2016, ArXiv.

[26]  Ming Li,et al.  Reversible simulation of irreversible computation , 1996, Proceedings of Computational Complexity (Formerly Structure in Complexity Theory).

[27]  Erik D. Demaine,et al.  Energy-Efficient Algorithms , 2016, ITCS.

[28]  D. Wolpert,et al.  Dependence of dissipation on the initial distribution over states , 2016, 1607.00956.

[29]  Alexander Schrijver,et al.  Cones of Matrices and Set-Functions and 0-1 Optimization , 1991, SIAM J. Optim..

[30]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[31]  L. Brillouin,et al.  Science and information theory , 1956 .

[32]  Thomas F. Knight,et al.  Reversibility in Optimally Scalable Computer Architectures , 1997 .

[33]  Hanif D. Sherali,et al.  A Hierarchy of Relaxations Between the Continuous and Convex Hull Representations for Zero-One Programming Problems , 1990, SIAM J. Discret. Math..

[34]  Richard E. Brown,et al.  Report to Congress on Server and Data Center Energy Efficiency: Public Law 109-431 , 2008 .

[35]  Farid Chejne,et al.  A simple derivation of crooks relation , 2013 .

[36]  M. Mitzenmacher,et al.  Probability and Computing: Chernoff Bounds , 2005 .

[37]  Shay Moran,et al.  Approximate Nonnegative Rank Is Equivalent to the Smooth Rectangle Bound , 2014, ICALP.

[38]  G. Dantzig.  Origins of the simplex method , 1990 .

[39]  M. Yannakakis.  Expressing combinatorial optimization problems by linear programs , 1991, Symposium on the Theory of Computing.

[40]  Jordan M. Horowitz,et al.  Thermodynamics with Continuous Information Flow , 2014, 1402.3276.

[41]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[42]  Ming Li,et al.  Reversible Simulation of Irreversible Computation by Pebble Games , 1997, ArXiv.

[43]  Christian Piguet,et al.  Why design reliable chips when faulty ones are even better , 2013, 2013 Proceedings of the ESSCIRC (ESSCIRC).

[44]  G. Crooks.  Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[45]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[46]  Oded Goldreich,et al.  On the theory of average case complexity , 1989, STOC '89.

[47]  Logan Kugler.  Is "good enough" computing good enough? , 2015, Commun. ACM.

[48]  C. Jarzynski,et al.  Information Processing and the Second Law of Thermodynamics: An Inclusive Hamiltonian Approach. , 2013, 1308.5001.

[49]  J. Lasserre.  Moments, Positive Polynomials And Their Applications , 2009 .

[50]  Shizume.  Heat generation required by information erasure. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[51]  Christos H. Papadimitriou,et al.  Reversible Simulation of Space-Bounded Computations , 1995, Theor. Comput. Sci..

[52]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[53]  David H. Wolpert,et al.  Extending Landauer's Bound from Bit Erasure to Arbitrary Computation , 2015, 1508.05319.

[54]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[55]  J. Lasserre.  An Introduction to Polynomial and Semi-Algebraic Optimization , 2015 .

[56]  Alan Mishchenko,et al.  Applying Logic Synthesis for Speeding Up SAT , 2007, SAT.

[57]  Rajeev Motwani,et al.  Randomized Algorithms , 1995, SIGA.

[58]  Woongki Baek,et al.  Green: a framework for supporting energy-conscious programming using controlled approximation , 2010, PLDI '10.

[59]  L. Szilard.  On the decrease of entropy in a thermodynamic system by the intervention of intelligent beings. , 1964, Behavioral science.

[60]  Leslie G. Valiant,et al.  Graph-Theoretic Properties in computational Complexity , 1976, J. Comput. Syst. Sci..

[61]  B. Codenotti.  Matrix Rigidity , 1999 .

[62]  H. P. Williams.  THEORY OF LINEAR AND INTEGER PROGRAMMING (Wiley-Interscience Series in Discrete Mathematics and Optimization) , 1989 .

[63]  Giovanni Manzini,et al.  Matrix rank and communication complexity , 2000 .

[64]  Charles H. Bennett,et al.  Logical reversibility of computation , 1973 .

[65]  C. Jarzynski.  Nonequilibrium Equality for Free Energy Differences , 1996, cond-mat/9610209.

[66]  U. Seifert.  Stochastic thermodynamics, fluctuation theorems and molecular machines , 2012, Reports on progress in physics. Physical Society.

[67]  Massimiliano Esposito,et al.  Second law and Landauer principle far from equilibrium , 2011, 1104.5165.

[68]  Seif Haridi,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[69]  Kenichi Morita,et al.  Reversible computing and cellular automata - A survey , 2008, Theor. Comput. Sci..

[70]  Masahito Ueda,et al.  Minimal energy cost for thermodynamic information processing: measurement and information erasure. , 2008, Physical review letters.

[71]  Ronald L. Rivest,et al.  Introduction to Algorithms, third edition , 2009 .

[72]  Luis Ceze,et al.  General-purpose code acceleration with limited-precision analog computation , 2014, 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA).

[73]  Warren P. Adams,et al.  A hierarchy of relaxation between the continuous and convex hull representations , 1990 .

[74]  H. Hasegawa,et al.  Generalization of the Second Law for a Nonequilibrium Initial State , 2009, 0907.1569.

[75]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[76]  M. Esposito,et al.  Three faces of the second law. I. Master equation formulation. , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[77]  Charles H. Bennett.  Time/Space Trade-Offs for Reversible Computation , 1989, SIAM J. Comput..

[78]  Kai Diethelm.  Tools for assessing and optimizing the energy requirements of high performance scientific computing software , 2016, ArXiv.

[79]  David K. Smith.  Theory of Linear and Integer Programming , 1987 .

[80]  Charles H. Bennett,et al.  Notes on Landauer's Principle, Reversible Computation, and Maxwell's Demon , 2002, physics/0210005.

[81]  Satyanarayana V. Lokam.  On the rigidity of Vandermonde matrices , 2000, Theor. Comput. Sci..

[82]  Paul M. B. Vitányi.  Time, space, and energy in reversible computing , 2005, CF '05.

[83]  Rolf Landauer,et al.  Minimal Energy Requirements in Communication , 1996, Science.

[84]  Bart Selman,et al.  Local search strategies for satisfiability testing , 1993, Cliques, Coloring, and Satisfiability.

[85]  R. Landauer.  Information is physical , 1991 .

[86]  Luca Trevisan,et al.  Average-Case Complexity , 2006, Found. Trends Theor. Comput. Sci..

[87]  Rolf Landauer,et al.  Irreversibility and heat generation in the computing process , 1961, IBM J. Res. Dev..

[88]  Alan T. Sherman,et al.  A Note on Bennett's Time-Space Tradeoff for Reversible Computation , 1990, SIAM J. Comput..

[89]  G. Crooks.  Nonequilibrium Measurements of Free Energy Differences for Microscopically Reversible Markovian Systems , 1998 .

[90]  David H. Wolpert,et al.  Number of hidden states needed to physically implement a given conditional distribution , 2017, New Journal of Physics.

[91]  T. Sagawa.  Thermodynamic and logical reversibilities revisited , 2013, 1311.1886.

[92]  Masahito Ueda,et al.  Fluctuation theorem with information exchange: role of correlations in stochastic thermodynamics. , 2012, Physical review letters.

[93]  Jean B. Lasserre,et al.  An Explicit Exact SDP Relaxation for Nonlinear 0-1 Programs , 2001, IPCO.

[94]  Ketan Mulmuley,et al.  Computational geometry : an introduction through randomized algorithms , 1993 .

[95]  Joan Boyar,et al.  Logic Minimization Techniques with Applications to Cryptology , 2013, Journal of Cryptology.