Random noise effects in pulse-mode digital multilayer neural networks

A pulse-mode digital multilayer neural network (DMNN) based on stochastic computing techniques is implemented with simple logic gates as basic computing elements. The pulse-mode signal representation and the use of simple logic gates for neural operations lead to a massively parallel yet compact and flexible network architecture, well suited for VLSI implementation. Algebraic neural operations are replaced by stochastic processes using pseudorandom pulse sequences. The distributions of the results from the stochastic processes are approximated using the hypergeometric distribution. Synaptic weights and neuron states are represented as probabilities and estimated as average pulse occurrence rates in corresponding pulse sequences. A statistical model of the noise (error) is developed to estimate the relative accuracy associated with stochastic computing in terms of mean and variance. Computational differences are then explained by comparison to deterministic neural computations. DMNN feedforward architectures are modeled in VHDL using character recognition problems as testbeds. Computational accuracy is analyzed, and the results of the statistical model are compared with the actual simulation results. Experiments show that the calculations performed in the DMNN are more accurate than those anticipated when Bernoulli sequences are assumed, as is common in the literature. Furthermore, the statistical model successfully predicts the accuracy of the operations performed in the DMNN.

[1]  G. Erten,et al.  A digital neural network architecture using random pulse trains , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[2]  Michael A. Shanblatt,et al.  A VLSI-based digital multilayer neural network architecture , 1993, [1993] Proceedings Third Great Lakes Symposium on VLSI-Design Automation of High Performance VLSI Systems.

[3]  Y. Suzuki,et al.  Digital systems for artificial neural networks , 1989, IEEE Circuits and Devices Magazine.

[4]  Kimmo Kaski,et al.  Pulse-density modulation technique in VLSI implementations of neural network algorithms , 1990 .

[5]  Alan F. Murray,et al.  Asynchronous VLSI neural networks using pulse-stream arithmetic , 1988 .

[6]  Alan F. Murray,et al.  Pulse-stream VLSI neural networks mixing analog and digital techniques , 1991, IEEE Trans. Neural Networks.

[7]  Jack L. Meador,et al.  Programmable impulse neural circuits , 1991, IEEE Trans. Neural Networks.

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  Max Stanford Tomlinson,et al.  A digital neural network architecture for VLSI , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[10]  Sergio Telles Ribeiro,et al.  Random-Pulse Machines , 1967, IEEE Trans. Electron. Comput..

[11]  Alan F. Murray,et al.  Integrated pulse stream neural networks: results, issues, and pointers , 1992, IEEE Trans. Neural Networks.

[12]  Mona E. Zaghloul,et al.  VLSI implementation of synaptic weighting and summing in pulse coded neural-type cells , 1992, IEEE Trans. Neural Networks.

[13]  Michael A. Shanblatt,et al.  Architecture and statistical model of a pulse-mode digital multilayer neural network , 1995, IEEE Trans. Neural Networks.

[14]  M. A. Shanblatt,et al.  An implementable digital multilayer neural network (DMNN) , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[15]  Roger Lipsett,et al.  VHSIC Hardware Description Language , 1985, Computer.