Energy aware algorithm design via probabilistic computing: from algorithms and models to Moore's law and novel (semiconductor) devices

The energy consumed by computations is becoming an increasing concern both within the context of high-performance systems well as embedded systems, on par with the past focus on raw speed or its derivative performance. In this talk, we will outline a novel framework for designing and analyzing algorithms wherein the figure of merit is the energy complexity – a measure of the (physical) energy consumed. Using the formulation of an energy-aware switch, and a network of such switches, fundamental limits will be established for the energy needed for switching deterministically, as well as energy savings derived from probabilistic switching, with a probability of being correct, p. Specifically, it is shown that a single deterministic switching step for computing a BIT consumes at least (.(κ.T ln(2))) Joules of energy, whereas the energy consumed by a single probabilistic switching step to compute a PBIT can be as low as (.(κ.T ln(2p))) Joules. These results are developed within the context of an idealized switching device introduced here, constrained by the laws of classical (statistical) thermodynamics (of Maxwell, Boltzmann and Gibbs), as well as by the constraints of idealized semiconductor devices. Based on this notion of switching, models for algorithm analysis and design, as well as upper- and lower- bounds on energy complexity and hence, for the first time, asymptotic energy savings via the use of probabilistic computing will be established. Possible approaches to realizing these probabilistic switches using conventional CMOS technology, as well as their potential for accelerating the current semiconductor roadmap that is based on deterministic computing, including the projections implied by Moore’s law, will be outlined. This work draws upon basic concepts from computer science, microelectronics and classical thermodynamics, and in the interest of being self-contained, the presentation will include a brief survey of the relevant thermodynamics.

[1]  Krishna V. Palem,et al.  Computational Proof as Experiment: Probabilistic Algorithms from a Thermodynamic Perspective , 2003, Verification: Theory and Practice.

[2]  R. Landauer,et al.  Irreversibility and heat generation in the computing process , 1961, IBM J. Res. Dev..

[3]  Richard J. Lipton,et al.  A Probabilistic Remark on Algebraic Program Testing , 1978, Inf. Process. Lett..

[4]  J. Schwartz,et al.  A note on monte carlo primality tests and algorithmic information theory , 1978 .

[5]  M. Rabin.  Degree of difficulty of computing a function and a partial ordering of recursive sets , 1960 .

[6]  Nicholas Pippenger,et al.  Developments in "The synthesis of reliable organ-isms from unreliable components , 1990 .

[7]  Donald Ervin Knuth,et al.  The Art of Computer Programming, 2nd Ed. (Addison-Wesley Series in Computer Science and Information , 1978 .

[8]  J. Hartmanis,et al.  On the Computational Complexity of Algorithms , 1965 .

[9]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[10]  Anthony J. G. Hey,et al.  Feynman Lectures on Computation , 1996 .

[11]  K.-U. Stein.  Noise-induced error rate as limiting factory for energy per operation in digital ICs , 1977 .

[12]  Jacob T. Schwartz,et al.  Fast Probabilistic Algorithms for Verification of Polynomial Identities , 1980, J. ACM.

[13]  James D. Meindl,et al.  Low power microelectronics: retrospect and prospect , 1995, Proc. IEEE.

[14]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[15]  L. Szilard.  On the decrease of entropy in a thermodynamic system by the intervention of intelligent beings. , 1964, Behavioral science.

[16]  J. von Neumann,et al.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[17]  A. Paz.  Probabilistic algorithms , 2003 .