The Correspondence Between Deterministic and Stochastic Digital Neurons: Analysis and Methodology

This paper analyzes the criteria for the direct correspondence between a deterministic neural network and its stochastic counterpart, and presents the guidelines that have been derived to establish such a correspondence during the design of a neural network application. In particular, the role of the slope and bias of the neuron activation function and that of the noise of its output have been addressed, thus filling a specific literature gap. This paper presents the results that have been theoretically derived in this regard, together with the simulations of few relevant application examples that have been performed to support them.

[1]  John Shawe-Taylor,et al.  Real time output derivatives for on chip learning using digital stochastic bit stream neurons , 1994 .

[2]  Nadia Nedjah,et al.  FPGA-based hardware architecture for neural networks: binary radix vs. stochastic , 2003, 16th Symposium on Integrated Circuits and Systems Design, 2003. SBCCI 2003. Proceedings..

[3]  Josep Freixas,et al.  The Greatest Allowed Relative Error in Weights and Threshold of Strict Separating Systems , 2008, IEEE Transactions on Neural Networks.

[4]  Hiroomi Hikawa,et al.  A digital hardware pulse-mode neuron with piecewise linear activation function , 2003, IEEE Trans. Neural Networks.

[5]  John Shawe-Taylor,et al.  Learning in Stochastic Bit Stream Neural Networks , 1996, Neural Networks.

[6]  Hiroomi Hikawa,et al.  A new digital pulse-mode neuron with adjustable activation function , 2003, IEEE Trans. Neural Networks.

[7]  Sing Kiong Nguang,et al.  Neural Network Implementation Using Bit Streams , 2007, IEEE Transactions on Neural Networks.

[8]  J. Shawe-Taylor,et al.  A stochastic neural architecture that exploits dynamically reconfigurable FPGAs , 1993, [1993] Proceedings IEEE Workshop on FPGAs for Custom Computing Machines.

[9]  Nadia Nedjah,et al.  Stochastic reconfigurable hardware for neural networks , 2003, Euromicro Symposium on Digital System Design, 2003. Proceedings..

[10]  John Shawe-Taylor,et al.  Stochastic bit-stream neural networks , 1999 .

[11]  Anthony G. Pipe,et al.  Implementing Spiking Neural Networks for Real-Time Signal-Processing and Control Applications: A Model-Validated FPGA Approach , 2007, IEEE Transactions on Neural Networks.

[12]  John Shawe-Taylor,et al.  Generating binary sequences for stochastic computing , 1994, IEEE Trans. Inf. Theory.

[13]  Jovisa D. Zunic,et al.  Separating Points by Parallel Hyperplanes— Characterization Problem , 2007, IEEE Transactions on Neural Networks.

[14]  Yasuji Sawada,et al.  Functional abilities of a stochastic logic neural network , 1992, IEEE Trans. Neural Networks.

[15]  Arthur Nádas,et al.  Binary classification by stochastic neural nets , 1995, IEEE Trans. Neural Networks.

[16]  Brad Hutchings,et al.  FPGA-based stochastic neural networks-implementation , 1994, Proceedings of IEEE Workshop on FPGA's for Custom Computing Machines.

[17]  Matteo Martincigh,et al.  A new architecture for digital stochastic pulse-mode neurons based on the voting circuit , 2005, IEEE Transactions on Neural Networks.

[18]  Howard C. Card,et al.  Stochastic Neural Computation I: Computational Elements , 2001, IEEE Trans. Computers.