Hardware-level Bayesian inference

Brain-inspired, inherently parallel computation has been proven to excel at tasks where the intrinsically serial Von Neumann architecture struggles. This has led to vast efforts aimed towards developing bio-inspired electronics, most notably in the guise of artificial neural networks (ANNs). However, ANNs are simply one possible substrate upon which computation can be carried out; their configuration determining what sort of computational function is being performed. In this work we show how Bayesian inference, a fundamental computational function, can be carried out using arrays of memristive devices, demonstrating computation directly using probability distributions as inputs and outputs. Our approach bypasses the need to map the Bayesian computation on an ANN (or any other) substrate since computation is carried out by simply providing the input distributions and letting Ohm’s law converge the voltages within the system to the correct answer. We show the fundamental circuit blocks used to enable this style of computation, examine how memristor non-idealities affect the quality of computation and exemplify a ‘Bayesian learning machine’ performing a simple task with no need for any digital arithmetic-logic operations.

[1]  Robert A. Legenstein,et al.  Ensembles of Spiking Neurons with Noise Support Optimal Probabilistic Inference in a Dynamically Changing Environment , 2014, PLoS Comput. Biol..

[2]  J Joshua Yang,et al.  Memristive devices for computing. , 2013, Nature nanotechnology.

[3]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[4]  R Stanley Williams,et al.  Sub-100 fJ and sub-nanosecond thermally driven threshold switching in niobium oxide crosspoint nanodevices , 2012, Nanotechnology.

[5]  Lawrence M. Murray Bayesian State-Space Modelling on High-Performance Hardware Using LibBi , 2013, 1306.3277.

[6]  L. Chua Memristor-The missing circuit element , 1971 .

[7]  S. Balatti,et al.  Resistive Switching by Voltage-Driven Ion Migration in Bipolar RRAM—Part II: Modeling , 2012, IEEE Transactions on Electron Devices.

[8]  Sally A. McKee,et al.  Reflections on the memory wall , 2004, CF '04.

[9]  R. Waser,et al.  Nanoionics-based resistive switching memories Many metal – insulator – , 2007 .

[10]  Ali Khiat,et al.  A compact Verilog-A ReRAM switching model , 2017, ArXiv.

[11]  Gert Cauwenberghs,et al.  Neuromorphic Silicon Neuron Circuits , 2011, Front. Neurosci.

[12]  Yong Zhu,et al.  Low-Power Wearable Systems for Continuous Monitoring of Environment and Health for Chronic Respiratory Disease , 2016, IEEE Journal of Biomedical and Health Informatics.

[13]  Fayez Gebali,et al.  Binary LNS-based naive Bayes inference engine for spam control: noise analysis and FPGA implementation , 2008, IET Comput. Digit. Tech..

[14]  Jin-Ki Kim,et al.  A 3.3 V 32 Mb NAND flash memory with incremental step pulse programming scheme , 1995, Proceedings ISSCC '95 - International Solid-State Circuits Conference.

[15]  T. Prodromakis,et al.  High Density Crossbar Arrays with Sub- 15 nm Single Cells via Liftoff Process Only , 2016, Scientific Reports.

[16]  K. Likharev CMOL : A silicon-based bottom-up approach to nanoelectronics , 2005 .

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Ali Khiat,et al.  An RRAM Biasing Parameter Optimizer , 2015, IEEE Transactions on Electron Devices.

[19]  Jens Sauerbrey,et al.  A 0.5-V 1-μW successive approximation ADC , 2003, IEEE J. Solid State Circuits.