CMOL/CMOS Implementations of Bayesian Polytree Inference: Digital and Mixed-Signal Architectures and Performance/Price

In this paper, we focus on aspects of the hardware implementation of the Bayesian inference framework within the George and Hawkins' model. This framework is based on Judea Pearl's belief propagation. We then present a ¿hardware design space exploration¿ methodology for implementing and analyzing the (digital and mixed-signal) hardware for the Bayesian (polytree) inference framework. This, particular, methodology involves: analyzing the computational/operational cost and the related microarchitecture, exploring candidate hardware components, proposing various custom architectures using both traditional CMOS and hybrid nanotechnology CMOS/nanowire/molecular hybrid (CMOL), and investigating the baseline performance/price of these hardware architectures. The results suggest that hybrid nanotechnology is a promising candidate to implement Bayesian inference. Such implementations utilize the very high density storage/computation benefits of these new nanoscale technologies much more efficiently, for example, the throughput per 858 mm2 obtained for CMOL-based architectures is 32-40 times better than the TPM for a CMOS based multiprocessor/multifield-programmable gate array system, and almost 2000 times better than the TPM for a single PC implementation. In general, the assessment of such hypothetical hardware architectures provides a baseline for large-scale implementations of Bayesian inference, and guidance for implementing the same using nanogrid structures.

[1]  Changjian Gao,et al.  Cortical Models Onto CMOL and CMOS— Architectures and Performance/Price , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[2]  K.K. Likharev,et al.  Reconfigurable Hybrid CMOS/Nanodevice Circuits for Image Processing , 2007, IEEE Transactions on Nanotechnology.

[3]  Gert Cauwenberghs,et al.  Charge-mode parallel architecture for vector-matrix multiplication , 2001 .

[4]  Tarek M. Taha,et al.  A neocortex model implementation on reconfigurable logic with streaming memory , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[5]  John N. Coleman,et al.  A 32 bit logarithmic arithmetic unit and its performance compared to floating-point , 1999, Proceedings 14th IEEE Symposium on Computer Arithmetic (Cat. No.99CB36336).

[6]  David Lewis.  114 MFLOPS logarithmic number system arithmetic unit for DSP applications , 1995 .

[7]  D. B. Davis,et al.  Intel Corp. , 1993 .

[8]  Boris Murmann.  Digitally Assisted Analog Circuits , 2006, IEEE Micro.

[9]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[10]  E. Rechtin,et al.  The art of systems architecting , 1996, IEEE Spectrum.

[11]  Xiaolong Ma,et al.  Architectures for nanoelectronic implementation of artificial neural networks: new results , 2005, Neurocomputing.

[12]  Tarek M. Taha,et al.  Hardware acceleration of image recognition through a visual cortex model , 2008 .

[13]  Massoud Pedram,et al.  Architectures for silicon nanoelectronics and beyond , 2007, Computer.

[14]  Johannes Schemmel,et al.  Wafer-scale integration of analog neural networks , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[15]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[16]  Tarek M. Taha,et al.  Scaling analysis of a neocortex inspired cognitive model on the Cray XD1 , 2008, The Journal of Supercomputing.

[17]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[18]  Jan M. Rabaey,et al.  Digital Integrated Circuits: A Design Perspective , 1995 .

[19]  Kwabena Boahen,et al.  Point-to-point connectivity between neuromorphic chips using address events , 2000 .

[20]  D. Strukov,et al.  Defect-tolerant architectures for nanoelectronic crossbar memories. , 2007, Journal of nanoscience and nanotechnology.

[21]  Yong-Surk Lee,et al.  Low cost floating point arithmetic unit design , 2002, Proceedings. IEEE Asia-Pacific Conference on ASIC,.

[22]  S. Tam,et al.  An electrically trainable artificial neural network (ETANN) with 10240 'floating gate' synapses , 1990, International 1989 Joint Conference on Neural Networks.

[23]  Karl S. Hemmert,et al.  A comparison of floating point and logarithmic number systems for FPGAs , 2005, 13th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM'05).

[24]  D. Strukov,et al.  CMOL: Devices, Circuits, and Architectures , 2006 .

[25]  David Harris,et al.  CMOS VLSI Design: A Circuits and Systems Perspective , 2004 .

[26]  D. Hammerstrom.  A Survey of Bio‐Inspired and Other Alternative Architectures , 2010 .

[27]  Richard Granger,et al.  Brain circuit implementation: High-precision computation from low-precision components , 2005 .

[28]  Dan Hammerstrom,et al.  Digital VLSI for neural networks , 1998 .

[29]  D. Hammerstrom,et al.  CMOL Based Cortical Models , 2007 .

[30]  Shekhar Y. Borkar,et al.  Electronics beyond nano-scale CMOS , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[31]  Mark Horowitz,et al.  A zero-overhead self-timed 160-ns 54-b CMOS divider , 1991 .

[32]  Hae-Seung Lee,et al.  A 2.5 V 12 b 5 MSample/s pipelined CMOS ADC , 1996, 1996 IEEE International Solid-State Circuits Conference. Digest of TEchnical Papers, ISSCC.

[33]  D. George,et al.  Hierarchical Temporal Memory Concepts , Theory , and Terminology , 2006 .

[34]  Changjian Gao,et al.  CMOS / CMOL architectures for spiking cortical column , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[35]  Anders Lansner,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[36]  Dan Hammerstrom,et al.  Bayesian Memory, a Possible Hardware Building Block for Intelligent Systems , 2008, AAAI Fall Symposium: Biologically Inspired Cognitive Architectures.

[37]  Changjian Gao.  Hardware architectures and implementations for associative memories---the building blocks of hierarchically distributed memories , 2008 .

[38]  D. George,et al.  A hierarchical Bayesian model of invariant pattern recognition in the visual cortex , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[39]  W. Lu,et al.  Programmable Resistance Switching in Nanoscale Two-terminal Devices , 2008 .

[40]  Valeriu Beiu.  Grand Challenges of Nanoelectronics and Possible Architectural Solutions: What Do Shannon, von Neumann, Kolmogorov, and Feynman Have to do with Moore , 2007, 37th International Symposium on Multiple-Valued Logic (ISMVL'07).

[41]  J. Hawkins,et al.  On Intelligence , 2004 .

[42]  Amine Bermak,et al.  An Efficient Digital VLSI Implementation of Gaussian Mixture Models-Based Classifier , 2006, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[43]  Keshab K. Parhi,et al.  VLSI digital signal processing systems , 1999 .

[44]  Dmitri B. Strukov,et al.  Prospects for the development of digital CMOL circuits , 2007, 2007 IEEE International Symposium on Nanoscale Architectures.

[45]  R. Williams,et al.  Nano/CMOS architectures using a field-programmable nanowire interconnect , 2007 .

[46]  J. Bailey,et al.  Why VLSI implementations of associative VLCNs require connection multiplexing , 1988, IEEE 1988 International Conference on Neural Networks.

[47]  Shekhar Y. Borkar,et al.  Tomorrow's analog: just dead or just different? , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[48]  D. Strukov,et al.  CMOL FPGA: a reconfigurable architecture for hybrid digital circuits with two-terminal nanodevices , 2005 .

[49]  Thomas Dean,et al.  Learning invariant features using inertial priors , 2006, Annals of Mathematics and Artificial Intelligence.