2009 Special Issue

Five new theorems and a stochastic learning algorithm show that noise can benefit threshold neural signal detection by reducing the probability of detection error. The first theorem gives a necessary and sufficient condition for such a noise benefit when a threshold neuron performs discrete binary signal detection in the presence of additive scale-family noise. The theorem allows the user to find the optimal noise probability density for several closed-form noise types that include generalized Gaussian noise. The second theorem gives a noise-benefit condition for more general threshold signal detection when the signals have continuous probability densities. The third and fourth theorems reduce this noise benefit to a weighted-derivative comparison of signal probability densities at the detection threshold when the signal densities are continuously differentiable and when the noise is symmetric and comes from a scale family. The fifth theorem shows how collective noise benefits can occur in a parallel array of threshold neurons even when an individual threshold neuron does not itself produce a noise benefit. The stochastic gradient-ascent learning algorithm can find the optimal noise value for noise probability densities that do not have a closed form.

[1]  Klaus Obermayer,et al.  Adaptation and Selective Information Transmission in the Cricket Auditory Neuron AN2 , 2008, PLoS Comput. Biol..

[2]  Gerald B. Folland,et al.  Real Analysis: Modern Techniques and Their Applications , 1984 .

[3]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[4]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[5]  Bernd Schürmann,et al.  Stochastic resonance in the mutual information between input and output spike trains of noisy central neurons , 1998 .

[6]  Bart Kosko,et al.  Adaptive stochastic resonance in noisy neurons based on mutual information , 2004, IEEE Transactions on Neural Networks.

[7]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[8]  John G. Proakis,et al.  Digital Communications , 1983 .

[9]  N G Stocks,et al.  Information transmission in parallel threshold arrays: suprathreshold stochastic resonance. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  Kurt Wiesenfeld,et al.  Stochastic resonance and the benefits of noise: from ice ages to crayfish and SQUIDs , 1995, Nature.

[11]  L. Gammaitoni,et al.  Stochastic resonance and the dithering effect in threshold physical systems. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[12]  William B Levy,et al.  Energy-Efficient Neuronal Computation via Quantal Synaptic Failures , 2002, The Journal of Neuroscience.

[13]  Ashok Patel,et al.  Stochastic Resonance in Continuous and Spiking Neuron Models with Levy Noise , 2008 .

[14]  Zhonghuai Hou,et al.  Internal noise enhanced detection of hormonal signal through intracellular calcium oscillations , 2005 .

[15]  Doron Lancet,et al.  Computational Capacity of an Odorant Discriminator: The Linear Separability of Curves , 2000, Neural Computation.

[16]  J. Proudfoot,et al.  Noise , 1931, The Indian medical gazette.

[17]  Ashok Patel,et al.  Quantizer noise benefits in nonlinear signal detection with alpha-stable channel noise , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  C. Pearce,et al.  Stochastic Resonance: From Suprathreshold Stochastic Resonance to Stochastic Signal Quantization , 2008 .

[19]  David Allingham,et al.  THE APPLICATION OF SUPRATHRESHOLD STOCHASTIC RESONANCE TO COCHLEAR IMPLANT CODING , 2002, The Random and Fluctuating World.

[20]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry, Expanded Edition , 1987 .

[21]  Robert Krupinski,et al.  Approximated fast estimator for the shape parameter of generalized Gaussian distribution for a small sample size , 2015 .

[22]  Valeriu Beiu,et al.  VLSI implementations of threshold logic-a comprehensive survey , 2003, IEEE Trans. Neural Networks.

[23]  Pramod K. Varshney,et al.  Reducing Probability of Decision Error Using Stochastic Resonance , 2006, IEEE Signal Processing Letters.

[24]  K. Obermayer,et al.  Optimal noise-aided signal transmission through populations of neurons. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  G. Temple The theory of generalized functions , 1955, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[26]  Christof Koch,et al.  Detecting and Estimating Signals in Noisy Cable Structures, I: Neuronal Noise Sources , 1999, Neural Computation.

[27]  N G Stocks,et al.  Generic noise-enhanced coding in neuronal arrays. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[28]  François Chapeau-Blondeau,et al.  Noise-enhanced nonlinear detector to improve signal detection in non-Gaussian noise , 2006 .

[29]  Bart Kosko,et al.  Robust stochastic resonance for simple threshold neurons. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[30]  Mark D. McDonnell,et al.  Pooling networks for a discrimination task: noise-enhanced detection , 2007, SPIE International Symposium on Fluctuations and Noise.

[31]  G. V. Anand,et al.  Design of detectors based on stochastic resonance , 2003, Signal Process..

[32]  J. White,et al.  Channel noise in neurons , 2000, Trends in Neurosciences.

[33]  W. Stacey,et al.  Stochastic resonance improves signal detection in hippocampal CA1 neurons. , 2000, Journal of neurophysiology.

[34]  Peter Hänggi,et al.  Stochastic resonance in biology. How noise can enhance detection of weak signals and help improve biological information processing. , 2002, Chemphyschem : a European journal of chemical physics and physical chemistry.

[35]  Frank Moss,et al.  TEMPERATURE DEPENDENCE AND THE ROLE OF INTERNAL NOISE IN SIGNAL TRANSDUCTION EFFICIENCY OF CRAYFISH MECHANORECEPTORS , 1995 .

[36]  Ashok Patel,et al.  Neural signal-detection noise benefits based on error probability , 2009, 2009 International Joint Conference on Neural Networks.

[37]  François Chapeau-Blondeau,et al.  Noise-enhanced performance for an optimal Bayesian estimator , 2004, IEEE Transactions on Signal Processing.

[38]  L. Pinneo On noise in the nervous system. , 1966, Psychological review.

[39]  A. Faisal,et al.  Noise in the nervous system , 2008, Nature Reviews Neuroscience.

[40]  Ashok Patel,et al.  Optimal Noise Benefits in Neyman–Pearson and Inequality-Constrained Statistical Signal Detection , 2009, IEEE Transactions on Signal Processing.

[41]  Ashok Patel,et al.  Stochastic resonance in noisy spiking retinal and sensory neuron models , 2005, Neural Networks.

[42]  Wei Wang,et al.  INTERNAL-NOISE-ENHANCED SIGNAL TRANSDUCTION IN NEURONAL SYSTEMS , 1997 .

[43]  S. Nadarajah A generalized normal distribution , 2005 .

[44]  Sadatsugu Sakane,et al.  Suprathreshold stochastic resonance in visual signal detection , 2008, Behavioural Brain Research.

[45]  François Chapeau-Blondeau,et al.  Constructive role of noise in signal detection from parallel arrays of quantizers , 2005, Signal Process..

[46]  Derek Abbott,et al.  Optimal information transmission in nonlinear arrays through suprathreshold stochastic resonance , 2006 .

[47]  S. Kay,et al.  Can detectability be improved by adding noise? , 2000, IEEE Signal Processing Letters.

[48]  S. Gazor,et al.  Speech probability distribution , 2003, IEEE Signal Processing Letters.

[49]  T J Sejnowski,et al.  Noise enhanced hormonal signal transduction through intracellular calcium oscillations. , 2001, Biophysical Chemistry.

[50]  François Chapeau-Blondeau,et al.  Stochastic resonance and improvement by noise in optimal detection strategies , 2005, Digit. Signal Process..

[51]  C. L. Nikias,et al.  Signal processing with alpha-stable distributions and applications , 1995 .

[52]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT.

[53]  A. Zemanian Distribution Theory and Transform Analysis; An Introduction to Generalized Functions, With Applications , 1965 .

[54]  Peter Auer,et al.  A learning rule for very simple universal approximators consisting of a single layer of perceptrons , 2008, Neural Networks.

[55]  L. M. Ward,et al.  Stochastic resonance and sensory information processing: a tutorial and review of application , 2004, Clinical Neurophysiology.

[56]  M. Grigoriu Applied Non-Gaussian Processes , 1995 .

[57]  B. Kosko,et al.  Adaptive stochastic resonance , 1998, Proc. IEEE.

[58]  K. Appenteng,et al.  Multimodal distribution of amplitudes of miniature and spontaneous EPSPs recorded in rat trigeminal motoneurones. , 1996, The Journal of physiology.

[59]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[60]  A. Bulsara,et al.  Stochastic resonance in a single neuron model: theory and analog simulation. , 1991, Journal of theoretical biology.

[61]  Lorenzo Bruzzone,et al.  Image thresholding based on the EM algorithm and the generalized Gaussian distribution , 2007, Pattern Recognit..

[62]  Idan Segev,et al.  Ion Channel Stochasticity May Be Critical in Determining the Reliability and Precision of Spike Timing , 1998, Neural Computation.

[63]  C. Rao,et al.  Control, exploitation and tolerance of intracellular noise , 2002, Nature.

[64]  Bart Kosko,et al.  Stochastic resonance in noisy threshold neurons , 2003, Neural Networks.