Minimising Contrastive Divergence in Noisy, Mixed-mode VLSI Neurons

This paper presents VLSI circuits with continuous-valued probabilistic behaviour realized by injecting noise into each computing unit(neuron). Interconnecting the noisy neurons forms a Continuous Restricted Boltzmann Machine (CRBM), which has shown promising performance in modelling and classifying noisy biomedical data. The Minimising-Contrastive-Divergence learning algorithm for CRBM is also implemented in mixed-mode VLSI, to adapt the noisy neurons' parameters on-chip.

[1]  Gert Cauwenberghs,et al.  An analog VLSI recurrent neural network learning a continuous-time trajectory , 1996, IEEE Trans. Neural Networks.

[2]  Alan F. Murray,et al.  Novelty detection using products of simple experts--a potential architecture for embedded systems , 2001, Neural Networks.

[3]  Stuart Haber,et al.  A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks , 1991 .

[4]  E. Vittoz.  MOS transistors operated in the lateral bipolar mode and their application in CMOS technology , 1983, IEEE Journal of Solid-State Circuits.

[5]  Alan F. Murray,et al.  Mixed-signal VLSI implementation of the Products of Experts' contrastive divergence learning scheme , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[6]  Hussein Chible.  Analog circuit for synapse neural networks VLSI implementation , 2000, ICECS 2000. 7th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.00EX445).

[7]  Brendan J. Frey,et al.  Continuous Sigmoidal Belief Networks Trained using Slice Sampling , 1996, NIPS.

[8]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[9]  Alan F. Murray,et al.  Continuous restricted Boltzmann machine with an implementable training algorithm , 2003 .

[10]  Yannis Tsividis,et al.  Floating voltage-controlled resistors in CMOS technology , 1982 .

[11]  Alan F. Murray,et al.  Adaptive Stochastic Classifier for Noisy pH-ISFET Measurements , 2003, ICANN.

[12]  G. Wegmann,et al.  Basic principles of accurate dynamic current mirrors , 1990 .