Rectification of correlation by a sigmoid nonlinearity

We investigated the normalized autocovariance (correlation coefficient) function of the output of an erf( ) function nonlinearity subject to non-zero mean Gaussian noise input. When the sigmoid is wide compared to the input, or the input mean is close to the midpoint of the sigmoid, the output correlation coefficient function is very close to the input correlation coefficient function. When the noise mean and variance are such that there is a significant probability of operating in the saturation region and the sigmoid is not too flat, the correlation coefficient of the output function is less than that of the input. This difference is much greater when the correlation coefficient is negative than when it is positive. The sigmoid partially rectifies the correlation coefficient function.The analysis does not depend on the spectral properties of the input noise. All that is required is that the input at times t and (t + τ) be jointly gaussian with the same mean and autocovariance. The analysis therefore applies equally well to the case of two identical sigmoids with jointly gaussian inputs. This correlational rectification could help explain the parameter sensitivity of “neural network” models. If biological neurons share this property it could explain why few negative correlations between spike trains-have been observed.

[1]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[2]  D. Kernell.  The Limits of Firing Frequency in Cat Lumbosacral Motoneurones Possessing Different Time Course of Afterhyperpolarization , 1965 .

[3]  Roy Leipnik,et al.  The effect of instantaneous nonlinear devices on cross-correlation , 1958, IRE Trans. Inf. Theory.

[4]  N. Wiener,et al.  A statistical analysis of synaptic excitation. , 1949, Journal of cellular and comparative physiology.

[5]  D. Kernell.  High-Frequency Repetitive Firing of Cat Lumbosacral Motoneurones Stimulated by Long-Lasting Injected Currents , 1965 .

[6]  S. Grossberg.  Contour Enhancement , Short Term Memory , and Constancies in Reverberating Neural Networks , 1973 .

[7]  R. Baum.  The correlation function of smoothly limited Gaussian noise , 1957 .

[8]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[9]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[10]  Leon N. Cooper,et al.  An overview of neural networks: early models to real world systems , 1990 .

[11]  Robert Price,et al.  A useful theorem for nonlinear devices having Gaussian inputs , 1958, IRE Trans. Inf. Theory.

[12]  A. Aertsen,et al.  Evaluation of neuronal connectivity: Sensitivity of cross-correlation , 1985, Brain Research.

[13]  S. Rice.  Mathematical analysis of random noise , 1944 .

[14]  W. Freeman.  Analog simulation of prepyriform cortex in the cat , 1968 .

[15]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[16]  Ralph M. Siegel,et al.  Non-linear dynamical system theory and primary visual cortical processing , 1990 .

[17]  W. Freeman,et al.  How brains make chaos in order to make sense of the world , 1987, Behavioral and Brain Sciences.

[18]  W. Little.  The existence of persistent states in the brain , 1974 .

[19]  Esther L. Sabban,et al.  Enhanced labeling of mitotic retinal cells with an intraocular [3H]thymidine injectio , 1990, Neuroscience Letters.

[20]  D. G. Watts,et al.  Spectral analysis and its applications , 1968 .

[21]  H. Boogaard,et al.  Stochastic formulation of neural interaction , 1985 .

[22]  Vasilis Z. Marmarelis,et al.  Analysis of Physiological Systems , 1978, Computers in Biology and Medicine.

[23]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[24]  Moshe Abeles,et al.  Corticonics: Neural Circuits of Cerebral Cortex , 1991 .

[25]  Apostolos Traganitis,et al.  The effect of a memoryless nonlinearity on the spectrum of a random process , 1977, IEEE Trans. Inf. Theory.

[26]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[27]  Purvis Bedenbaugh,et al.  Plasticity in the rat somatosensory cortex induced by local microstimulation and theoretical investigations of information flow through neurons , 1993 .

[28]  W. Rall,et al.  Experimental monosynaptic input-output relations in the mammalian spinal cord. , 1955, Journal of cellular and comparative physiology.