Dynamic range and error tolerance of stochastic neural rate codes

Abstract An investigation is made of the statistical properties of unary stochastic rate codes, as employed in the implementation of pulsed neural networks. The dynamic range exhibits a reduced dependence on the representation time due to increased estimation errors from the variance in the stochastic encodings of the integers. At the same time, these codes exhibit improved tolerance to noise-induced bit errors. Comparisons are made with conventional integer arithmetic and with deterministic unary codes for neural networks, up to bit error rates as large as 0.3.

[1]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[2]  John Shawe-Taylor,et al.  Probabilistic Bit Stream Neural Chip: Theory , 1991 .

[3]  Michael A. Shanblatt,et al.  Architecture and statistical model of a pulse-mode digital multilayer neural network , 1995, IEEE Trans. Neural Networks.

[4]  Alan F. Murray,et al.  Pulse-based computation in VLSI neural networks , 1999 .

[5]  Antonio Torralba,et al.  Two digital circuits for a fully parallel stochastic neural network , 1995, IEEE Trans. Neural Networks.

[6]  Leopoldo García Franquelo,et al.  Fully parallel stochastic computation architecture , 1996, IEEE Trans. Signal Process..

[7]  Douglas S. Reeves,et al.  The TInMANN VLSI chip , 1992, IEEE Trans. Neural Networks.

[8]  Kishor S. Trivedi.  Probability and Statistics with Reliability, Queuing, and Computer Science Applications , 1984 .

[9]  Howard C. Card,et al.  Parallel Random Number Generation for VLSI Systems Using Cellular Automata , 1989, IEEE Trans. Computers.

[10]  John Shawe-Taylor,et al.  Learning in Stochastic Bit Stream Neural Networks , 1996, Neural Networks.

[11]  Kimmo Kaski,et al.  Pulse-density modulation technique in VLSI implementations of neural network algorithms , 1990 .

[12]  Howard C. Card,et al.  Stochastic arithmetic implementations of neural networks with in situ learning , 1993, IEEE International Conference on Neural Networks.

[13]  Howard C. Card,et al.  Stochastic Neural Computation I: Computational Elements , 2001, IEEE Trans. Computers.

[14]  Max Stanford Tomlinson,et al.  A digital neural network architecture for VLSI , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[15]  John Shawe-Taylor,et al.  Stochastic bit-stream neural networks , 1999 .

[16]  Max Stanford Tomlinson,et al.  DNNA: A Digital Neural Network Architecture , 1990 .

[17]  Yasuji Sawada,et al.  Functional abilities of a stochastic logic neural network , 1992, IEEE Trans. Neural Networks.

[18]  H. Card.  Stochastic rate and temporal codes with asymmetric bit errors , 2001 .

[19]  Thomas K. Miller,et al.  A digital architecture employing stochasticism for the simulation of Hopfield neural nets , 1989 .

[20]  Brian R. Gaines,et al.  Stochastic Computing Systems , 1969 .