Low power logic for statistical inference

Efficient hardware implementations of statistical inference continue to grow in importance for a wide range of computing applications. While CPU cycles are increasingly being used for statistical inference, transistors are also becoming increasingly statistical. For implementing statistical algorithms, could it be that statistical electronic substrates are a feature rather than a bug? We show that inference models can often be built from local constraints, and explain the gate-level mathematical functions required for the resulting inference solver. We suggest that signals should consist of probabilistic populations of particles representing samples from a probability distribution, with gate functions acting to transform these ensembles. Using this mapping from statistical physics to statistical inference, we present Bayesian logic circuits as highly efficient alternatives to digital standard cell libraries. For particular inference computations, novel VLSI architectures based on Bayesian logic circuits consume orders of magnitude less power and silicon area compared to conventional digital processors.

[1]  N. Metropolis,et al.  The Monte Carlo method. , 1949 .

[2]  R. Kikuchi A Theory of Cooperative Phenomena , 1951 .

[3]  J. von Neumann,et al.  Probabilistic Logic and the Synthesis of Reliable Organisms from Unreliable Components , 1956 .

[4]  Shmuel Winograd,et al.  Coding for Logical Operations , 1962, IBM J. Res. Dev..

[5]  B. Gilbert Translinear circuits: a proposed classification , 1975 .

[6]  Alan F. Murray,et al.  A Novel Computational and Signalling Method for VLSI Neural Networks , 1987, ESSCIRC '87: 13th European Solid-State Circuits Conference.

[7]  Alan F. Murray,et al.  VLSI neural networks , 1988 .

[8]  Alan F. Murray,et al.  VLSI for Parallel Processing, IEE Colloquium on , 1988 .

[9]  D. Arrowsmith Stan Ulam , 1989, Nature.

[10]  M. V. Fischetti,et al.  Monte Carlo simulation of a 30 nm dual-gate MOSFET: how short can Si go? , 1992, 1992 International Technical Digest on Electron Devices Meeting.

[11]  T. P. Caudell,et al.  Neural network architecture for linear programming , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[12]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[13]  Mohammed Benaissa,et al.  Fault-tolerant linear convolution using residue number systems , 1994, Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS '94.

[14]  Andrzej Cichocki,et al.  A new neural network for solving linear programming problems , 1996 .

[15]  Richard J. Carter,et al.  The Teramac Custom Computer: Extending the Limits with Defect Tolerance , 1996, DFT.

[16]  Daniel A. Spielman,et al.  Highly fault-tolerant parallel computation , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[17]  David J. C. MacKay,et al.  Low-density parity check codes over GF(q) , 1998, IEEE Communications Letters.

[18]  Benjamin William Vigoda,et al.  A nonlinear dynamic system for spread spectrum code acquisition , 1999 .

[19]  Felix Lustenberger,et al.  On the design of analog VLSI iterative decoders , 2000 .

[20]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[21]  K. Uchida,et al.  Single-electron random-number generator (RNG) for highly secure ubiquitous computing applications , 2002, Digest. International Electron Devices Meeting,.

[22]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[23]  Joseph Y. Halpern,et al.  Updating Probabilities , 2002, UAI.

[24]  Majid Ahmadi,et al.  A Number System with Continuous Valued Digits and Modulo Arithmetic , 2002, IEEE Trans. Computers.

[25]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[26]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[27]  Benjamin Vigoda,et al.  Analog Logic: Continuous-Time Analog Circuits for Statistical Signal Processing , 2003 .

[28]  James Tschanz,et al.  Parameter variations and impact on circuits and microarchitecture , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[29]  Neil Gershenfeld,et al.  Continuous-time analog circuits for statistical signal processing , 2003 .

[30]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[31]  Alexander Astaras,et al.  Pulse-stream binary stochastic hardware for neural computation : the Helmholtz Machine , 2004 .

[32]  N. Cerf,et al.  Intramolecular Hamiltonian logic gates , 2004 .

[33]  Edward J. McCluskey,et al.  Reconfigurable architecture for autonomous self-repair , 2004, IEEE Design & Test of Computers.

[34]  W.J. Gross,et al.  Stochastic Implementation of LDPC Decoders , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[35]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[36]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[37]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[38]  Michael Chertkov,et al.  Loop Calculus in Statistical Physics and Information Science , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  A. Asenov,et al.  Fermi-Dirac Statistics in Monte Carlo Simulations of InGaAs MOSFETs , 2006 .

[40]  Shie Mannor,et al.  Survey of Stochastic Computation on Factor Graphs , 2007, 37th International Symposium on Multiple-Valued Logic (ISMVL'07).

[41]  A. Fujiwara,et al.  Single-electron counting statistics and its circuit application in nanoscale field-effect transistors at room temperature. , 2009, Nanotechnology.

[42]  Stefano Pironio,et al.  Random numbers certified by Bell’s theorem , 2009, Nature.

[43]  Yee Whye Teh,et al.  Dirichlet Process , 2017, Encyclopedia of Machine Learning and Data Mining.

[44]  Douglas L. Jones,et al.  Stochastic computation , 2010, Design Automation Conference.

[45]  Hai Wei,et al.  ACCNT: A Metallic-CNT-Tolerant Design Methodology for Carbon Nanotube VLSI: Analyses and Design Guidelines , 2010, IEEE Transactions on Electron Devices.

[46]  W. Marsden I and J , 2012 .

[47]  N. Metropolis THE BEGINNING of the MONTE CARLO METHOD , 2022 .