Temporal code versus rate code for binary Information Sources

Neuroscientists formulate very different hypotheses about the nature of neural coding. At one extreme, it has been argued that neurons encode information through relatively slow changes in the arrival rates of individual spikes (rate codes) and that the irregularity in the spike trains reflects the noise in the system. At the other extreme, this irregularity is the code itself (temporal codes) so that the precise timing of every spike carries additional information about the input. It is well known that in the estimation of Shannon Information Transmission Rate, the patterns and temporal structures are taken into account, while the rate code is already determined by the firing rate, i.e. by the spike frequency. In this paper we compare these two types of codes for binary Information Sources, which model encoded spike trains. Assuming that the information transmitted by a neuron is governed by an uncorrelated stochastic process or by a process with a memory, we compare the Information Transmission Rates carried by such spike trains with their firing rates. Here we show that a crucial role in the relation between information transmission and firing rates is played by a factor that we call the jumping parameter. This parameter corresponds to the probability of transitions from the no-spike-state to the spike-state and vice versa. For low jumping parameter values, the quotient of information and firing rates is a monotonically decreasing function of the firing rate, and there therefore a straightforward, one-to-one, relation between temporal and rate codes. However, it turns out that for large enough values of the jumping parameter this quotient is a non-monotonic function of the firing rate and it exhibits a global maximum, so that in this case there is an optimal firing rate. Moreover, there is no one-to-one relation between information and firing rates, so the temporal and rate codes differ qualitatively. This leads to the observation that the behavior of the quotient of information and firing rates for a large jumping parameter value is especially important in the context of bursting phenomena. HighlightsTemporal code and firing rate code are compared for two types of Information Sources.Coefficient combining information with energy was proposed.Parameter which determines relation between both codes was found.Large spike train variability leads to qualitative difference between the codes.

[1]  D. Sherrington Stochastic Processes in Physics and Chemistry , 1983 .

[2]  Noelia Montejo,et al.  Information content in cortical spike trains during brain state transitions , 2013, Journal of sleep research.

[3]  Adelbert Ames,et al.  CNS energy metabolism as related to function , 2000, Brain Research Reviews.

[4]  Kay Chen Tan,et al.  A brain-inspired spiking neural network model with temporal encoding and learning , 2014, Neurocomputing.

[5]  Silvestro Micera,et al.  How central inputs and force and velocity feedbacks determine motoneurons activity during voluntary hand movements , 2015, BMC Neuroscience.

[6]  T. Sejnowski,et al.  23 problems in systems neuroscience , 2006 .

[7]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[8]  Wulfram Gerstner,et al.  Neuronal Dynamics: From Single Neurons To Networks And Models Of Cognition , 2014 .

[9]  Qingxiang Wu,et al.  Learning under weight constraints in networks of temporal encoding spiking neurons , 2006, Neurocomputing.

[10]  G. Stanley Reading and writing the neural code , 2013, Nature Neuroscience.

[11]  H. Spitzer,et al.  Temporal encoding of two-dimensional patterns by single units in primate primary visual cortex. I. Stimulus-response relations. , 1990, Journal of neurophysiology.

[12]  Robert E. Kass,et al.  A Spike-Train Probability Model , 2001, Neural Computation.

[13]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[14]  William Bialek,et al.  Entropy and Information in Neural Spike Trains , 1996, cond-mat/9603127.

[15]  L. Deng,et al.  The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] , 2012, IEEE Signal Processing Magazine.

[16]  Allan D. Coop,et al.  Deciphering the neural code: neuronal discharge variability is preferentially controlled by the temporal distribution of afferent impulses , 2001, Neurocomputing.

[17]  Alexander Borst,et al.  Information theory and neural coding , 1999, Nature Neuroscience.

[18]  A. C. Webb,et al.  The spontaneous activity of neurones in the cat’s cerebral cortex , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[19]  L. Ricciardi,et al.  The Ornstein-Uhlenbeck process as a model for neuronal activity , 1979, Biological Cybernetics.

[20]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[21]  Alicia D'Anjou,et al.  Metabolic efficiency with fast spiking in the squid axon , 2012, Front. Comput. Neurosci..

[22]  C. Koch,et al.  Encoding of visual information by LGN bursts. , 1999, Journal of neurophysiology.

[23]  Paul F. M. J. Verschure,et al.  Stimulus encoding during the early stages of olfactory processing: A modeling study using an artificial olfactory system , 2001, Neurocomputing.

[24]  Janusz Szczepanski,et al.  Mutual information against correlations in binary communication channels , 2015, BMC Neuroscience.

[25]  E. Adrian,et al.  The impulses produced by sensory nerve‐endings , 1926 .

[26]  Haizhou Li,et al.  Rapid Feedforward Computation by Temporal Encoding and Learning With Spiking Neurons , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[28]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[29]  E. Adrian,et al.  The impulses produced by sensory nerve endings , 1926, The Journal of physiology.

[30]  Chun-I Yeh,et al.  Temporal precision in the neural code and the timescales of natural vision , 2007, Nature.

[31]  A V Herz,et al.  Neural codes: firing rates and beyond. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[32]  James Ting-Ho Lo,et al.  A cortex-like learning machine for temporal hierarchical pattern clustering, detection, and recognition , 2012, Neurocomputing.

[33]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[34]  Abraham Lempel,et al.  On the Complexity of Finite Sequences , 1976, IEEE Trans. Inf. Theory.

[35]  Kazuyuki Aihara,et al.  Duality of Rate Coding and Temporal Coding in Multilayered Feedforward Networks , 2003, Neural Computation.

[36]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[37]  Heinrich H. Bülthoff,et al.  Learned Non-Rigid Object Motion is a View-Invariant Cue to Recognizing Novel Objects , 2012, Front. Comput. Neurosci..

[38]  C. Cavallero,et al.  The quest for dream sources , 1993 .

[39]  Janusz Szczepanski,et al.  How do the amplitude fluctuations affect the neuronal transmission efficiency , 2013, Neurocomputing.

[40]  José María Amigó,et al.  Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity , 2004, Neural Computation.

[41]  Yuri M. Suhov,et al.  Nonparametric Entropy Estimation for Stationary Processesand Random Fields, with Applications to English Text , 1998, IEEE Trans. Inf. Theory.

[42]  R. Stein A THEORETICAL ANALYSIS OF NEURONAL VARIABILITY. , 1965, Biophysical journal.

[43]  E. Kaplan,et al.  Dynamics of neurons in the cat lateral geniculate nucleus: in vivo electrophysiology and computational modeling. , 1995, Journal of neurophysiology.

[44]  Sundaram Suresh,et al.  Development of a Self-Regulating Evolving Spiking Neural Network for classification problem , 2016, Neurocomputing.

[45]  Simon J. Thorpe,et al.  Sparse spike coding in an asynchronous feed-forward multi-layer neural network using matching pursuit , 2004, Neurocomputing.

[46]  Per Jesper Sjöström,et al.  Novel presynaptic mechanisms for coincidence detection in synaptic plasticity , 2006, Current Opinion in Neurobiology.

[47]  Haizhou Li,et al.  A Spike-Timing-Based Integrated Model for Pattern Recognition , 2013, Neural Computation.

[48]  Vito Di Maio,et al.  Regulation of information passing by synaptic transmission: A short review , 2008, Brain Research.

[49]  Andreas G. Andreou,et al.  Spike communication of dynamic stimuli: rate decoding versus temporal decoding , 2004, Neurocomputing.

[50]  N. Kampen,et al.  Stochastic processes in physics and chemistry , 1981 .

[51]  Karsten Keller,et al.  Ordinal symbolic analysis and its application to biomedical recordings , 2015, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[52]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[53]  Laura Sacerdote,et al.  The Ornstein–Uhlenbeck neuronal model with signal-dependent noise , 2001 .