Shot-noise-limited performance of optical neural networks

The performance of neural networks for which weights and signals are modeled by shot-noise processes is considered. Examples of such networks are optical neural networks and biological systems. We develop a theory that facilitates the computation of the average probability of error in binary-input/binary-output multistage and recurrent networks. We express the probability of error in terms of two key parameters: the computing-noise parameter and the weight-recording-noise parameter. The former is the average number of particles per clock cycle per signal and it represents noise due to the particle nature of the signal. The latter represents noise in the weight-recording process and is the average number of particles per weight. For a fixed computing-noise parameter, the probability of error decreases with the increase in the recording-noise parameter and saturates at a level limited by the computing-noise parameter. A similar behavior is observed when the role of the two parameters is interchanged. As both parameters increase, the probability of error decreases to zero exponentially fast at a rate that is determined using large deviations. We show that the performance can be optimized by a selective choice of the nonlinearity threshold levels. For recurrent networks, as the number of iterations increases, the probability of error increases initially and then saturates at a level determined by the stationary distribution of a Markov chain.

[1]  Donald L. Snyder,et al.  Random Point Processes in Time and Space , 1991 .

[2]  R. Haber,et al.  Visual Perception , 2018, Encyclopedia of Database Systems.

[3]  Bernard Widrow,et al.  Sensitivity of feedforward neural networks to weight errors , 1990, IEEE Trans. Neural Networks.

[4]  Richard P. Lippmann,et al.  LNKnet: Neural Network, Machine-Learning, and Statistical Software for Pattern Classification , 1993 .

[5]  B. Saleh CHAPTER 10 – Quantum Noise in Optical Processing , 1994 .

[7]  R. Ellis,et al.  LARGE DEVIATIONS FOR A GENERAL-CLASS OF RANDOM VECTORS , 1984 .

[8]  Bahaa E. A. Saleh,et al.  Quantum Fluctuations and Adaptive Spatial Interactions in the Visual System , 1978, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Marek Elbaum,et al.  Perceptrons for photon-limited image classification , 1992, Defense, Security, and Sensing.

[10]  Samuel Karlin,et al.  A First Course on Stochastic Processes , 1968 .

[11]  G. Parmigiani Large Deviation Techniques in Decision, Simulation and Estimation , 1992 .

[12]  A F Gmitro,et al.  Statistical performance of outer-product associative memory models. , 1989, Applied optics.

[13]  Lin Zhang,et al.  Impact of noise on an optoelectronic neural network , 1992, Defense, Security, and Sensing.

[14]  John F. Walkup,et al.  Noise effects and fault tolerance in hopfield-type neural networks , 1990 .

[15]  D Psaltis,et al.  Optical implementation of the Hopfield model. , 1985, Applied optics.