Architecture and statistical model of a pulse-mode digital multilayer neural network

A new architecture and a statistical model for a pulse-mode digital multilayer neural network (DMNN) are presented. Algebraic neural operations are replaced by stochastic processes using pseudo-random pulse sequences. Synaptic weights and neuron states are represented as probabilities and estimated as average rates of pulse occurrences in corresponding pulse sequences. A statistical model of error (or noise) is developed to estimate relative accuracy associated with stochastic computing in terms of mean and variance. The stochastic computing technique is implemented with simple logic gates as basic computing elements leading to a high neuron-density on a chip. Furthermore, the use of simple logic gates for neural operations, the pulse-mode signal representation, and the modular design techniques lead to a massively parallel yet compact and flexible network architecture, well suited for VLSI implementation. Any size of a feedforward network can be configured where processing speed is independent of the network size. Multilayer feedforward networks are modeled and applied to pattern classification problems such as encoding and character recognition.

[1]  Max Stanford Tomlinson,et al.  A digital neural network architecture for VLSI , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[2]  Carver A. Mead,et al.  A novel associative memory implemented using collective computation , 1990 .

[3]  Alan F. Murray,et al.  Integrated pulse stream neural networks: results, issues, and pointers , 1992, IEEE Trans. Neural Networks.

[4]  Alan F. Murray,et al.  Asynchronous arithmetic for VLSI neural systems , 1987 .

[5]  Michael A. Shanblatt,et al.  Random noise effects in pulse-mode digital multilayer neural networks , 1995, IEEE Trans. Neural Networks.

[6]  Solomon W. Golomb,et al.  Shift Register Sequences , 1981 .

[7]  Alan F. Murray,et al.  Pulse-stream VLSI neural networks mixing analog and digital techniques , 1991, IEEE Trans. Neural Networks.

[8]  David E. van den Bout,et al.  A stochastic architecture for neural nets , 1988, IEEE 1988 International Conference on Neural Networks.

[9]  Stuart Haber,et al.  A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks , 1991 .

[10]  Lawrence D. Jackel,et al.  VLSI implementation of a neural network memory with several hundreds of neurons , 1987 .

[11]  M. A. Shanblatt,et al.  An implementable digital multilayer neural network (DMNN) , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[12]  Y. Hirai,et al.  A digital neuro-chip with unlimited connectability for large scale neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[13]  David E. van den Bout,et al.  The VLSI implementation of STONN , 1990, 1990 IJCNN International Joint Conference on Neural Networks.