Neural timing nets

Formulations of artificial neural networks are directly related to assumptions about neural coding in the brain. Traditional connectionist networks assume channel-based rate coding, while time-delay networks convert temporally-coded inputs into rate-coded outputs. Neural timing nets that operate on time structured input spike trains to produce meaningful time-structured outputs are proposed. Basic computational properties of simple feedforward and recurrent timing nets are outlined and applied to auditory computations. Feed-forward timing nets consist of arrays of coincidence detectors connected via tapped delay lines. These temporal sieves extract common spike patterns in their inputs that can subserve extraction of common fundamental frequencies (periodicity pitch) and common spectrum (timbre). Feedforward timing nets can also be used to separate time-shifted patterns, fusing patterns with similar internal temporal structure and spatially segregating different ones. Simple recurrent timing nets consisting of arrays of delay loops amplify and separate recurring time patterns. Single- and multichannel recurrent timing nets are presented that demonstrate the separation of concurrent, double vowels. Timing nets constitute a new and general neural network strategy for performing temporal computations on neural spike trains: extraction of common periodicities, detection of recurring temporal patterns, and formation and separation of invariant spike patterns that subserve auditory objects.

[1]  S. W. Beet,et al.  Visual representations of speech signals , 1993 .

[2]  R. Patterson,et al.  Time-domain modeling of peripheral auditory processing: a modular architecture and a software platform. , 1995, The Journal of the Acoustical Society of America.

[3]  Steven Greenberg,et al.  Computational Models of Auditory Function , 2001 .

[4]  T. Gelder,et al.  Mind as Motion: Explorations in the Dynamics of Cognition , 1995 .

[5]  E. Boring Sensation and Perception. (Scientific Books: Sensation and Perception in the History of Experimental Psychology) , 1943 .

[6]  E. John,et al.  Foundations of Cognitive Processes , 1977 .

[7]  P. Cariani Temporal Coding of Periodicity Pitch in the Auditory System: An Overview , 1999, Neural plasticity.

[8]  Stephen Grossberg,et al.  Neural dynamics of motion perception, recognition learning, and spatial attention , 1996 .

[9]  Peter Cariani,et al.  Temporal coding of sensory information in the brain , 2001 .

[10]  V. Mountcastle,et al.  The problem of sensing and the neural coding of sensory events , 1967 .

[11]  P. Cariani Emergence of new signal-primitives in neural systems , 1997 .

[12]  Peter Cariani,et al.  Temporal Codes, Timing Nets, and Music Perception , 2001 .

[13]  C E Carr,et al.  Processing of temporal information in the brain. , 1993, Annual review of neuroscience.

[14]  V. Braitenberg Is the cerebellar cortex a biological clock in the millisecond range? , 1967, Progress in brain research.

[15]  J. Kauer Response patterns of amphibian olfactory bulb neurones to odour stimulation , 1974, The Journal of physiology.

[16]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[17]  Q Summerfield,et al.  Perception of concurrent vowels: effects of harmonic misalignment and pitch-period asynchrony. , 1991, The Journal of the Acoustical Society of America.

[18]  E. F. Evans,et al.  Psychophysics and Physiology of Hearing , 1979 .

[19]  B. Delgutte,et al.  Neural correlates of the pitch of complex tones. II. Pitch shift, pitch ambiguity, phase invariance, pitch circularity, rate pitch, and the dominance region for pitch. , 1996, Journal of neurophysiology.

[20]  E. Boring Sensation and Perception. (Scientific Books: Sensation and Perception in the History of Experimental Psychology) , 1943 .

[21]  Wolf Singer,et al.  Time as Coding Space in Neocortical Processing: A Hypothesis , 1994 .

[22]  B. Delgutte,et al.  Neural correlates of the pitch of complex tones. I. Pitch and pitch salience. , 1996, Journal of neurophysiology.

[23]  W. Singer,et al.  Temporal Coding in the Brain , 1994, Research and Perspectives in Neurosciences.

[24]  E. Owens,et al.  An Introduction to the Psychology of Hearing , 1997 .

[25]  Günter Ehret,et al.  The Central Auditory System , 1996 .

[26]  J. Culling,et al.  Dichotic pitches as illusions of binaural unmasking. I. Huggins' pitch and the "binaural edge pitch". , 1998, The Journal of the Acoustical Society of America.

[27]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[28]  Manfred Clynes,et al.  Music, Mind, and Brain , 1982, Springer US.

[29]  Peter Cariani,et al.  Temporal coding of sensory information , 1997 .

[30]  W. McCulloch,et al.  The limiting information capacity of a neuronal link , 1952 .

[31]  P. D. Di Lorenzo,et al.  Perceptual consequences of electrical stimulation in the gustatory system. , 1993, Behavioral neuroscience.

[32]  M. P. Friedman,et al.  HANDBOOK OF PERCEPTION , 1977 .

[33]  Nathaniel I. Durlach,et al.  Chapter 11 – MODELS OF BINAURAL INTERACTION , 1978 .

[34]  E. Vaadia,et al.  Synchronization in neuronal transmission and its importance for information processing. , 1994 .

[35]  Richard Durbin,et al.  The computing neuron , 1989 .

[36]  J. Eggermont The Correlative Brain: Theory and Experiment in Neural Interaction , 1990 .

[37]  R. Meddis,et al.  A unitary model of pitch perception. , 1997, The Journal of the Acoustical Society of America.

[38]  H. C. Longuet-Higgins,et al.  Mental Processes: Studies in Cognitive Science , 1987 .

[39]  Sigmund Koch Sensory, perceptual, and physiological formulations , 1959 .

[40]  G S Wasserman,et al.  Isomorphism, task dependence, and the multiple meaning theory of neural coding. , 1992, Biological signals.

[41]  A. de Cheveigné Cancellation model of pitch perception. , 1998, The Journal of the Acoustical Society of America.

[42]  Michaël Titus Maria Scheffers,et al.  Sifting vowels. Auditory pitch analysis and sound segregation. , 1983 .

[43]  Rodney Cotterill,et al.  Models of brain function , 1989 .

[44]  Richard F. Lyon,et al.  On the importance of time—a temporal representation of sound , 1993 .

[45]  U. Tilmann Zwicker,et al.  Auditory recognition of diotic and dichotic vowel pairs , 1984, Speech Commun..

[46]  J. Lettvin,et al.  Multiple meaning in single visual units. , 1970, Brain, behavior and evolution.

[47]  I. Whitfield Discharge Patterns of Single Fibers in the Cat's Auditory Nerve , 1966 .

[48]  Donald O. Walter,et al.  Self-Organizing Systems , 1987, Life Science Monographs.

[49]  Robert F. Port,et al.  Neural Representation of Temporal Patterns , 1995, Springer US.

[50]  Ralph R. Miller,et al.  The Role of Time in Elementary Associations , 1993 .

[51]  Peter Cariani,et al.  Neural Representation of Pitch through Temporal Autocorrelation , 1997 .

[52]  Stephanie Seneff,et al.  Pitch and spectral analysis of speech based on an auditory synchrony model , 1985 .

[53]  L A JEFFRESS,et al.  A place theory of sound localization. , 1948, Journal of comparative and physiological psychology.

[54]  H J Reitboeck,et al.  Color-dependent distribution of spikes in single optic tract fibers of the cat. , 1974, Vision research.

[55]  R. Meddis,et al.  Virtual pitch and phase sensitivity of a computer model of the auditory periphery. II: Phase sensitivity , 1991 .

[56]  James M. Bower,et al.  Computational Neuroscience: Trends in Research , 1996 .

[57]  Christopher Longuet-Higgins A mechanism for the storage of temporal correlations , 1989 .

[58]  M. Sanders Handbook of Sensory Physiology , 1975 .

[59]  Ray Meddis,et al.  Virtual pitch and phase sensitivity of a computer model of the auditory periphery , 1991 .

[60]  H. Steven Colburn,et al.  Computational Models of Binaural Processing , 1996 .

[61]  V. Braitenberg Functional Interpretation of Cerebellar Histology , 1961, Nature.

[62]  Leon van Noorden,et al.  TWO CHANNEL PITCH PERCEPTION , 1982 .

[63]  Oded Ghitza,et al.  Temporal non-place information in the auditory-nerve firing patterns as a front-end for speech recognition in a noisy environment , 1988 .

[64]  Michael Feuerstein,et al.  Principles of Psychophysiology , 1986 .

[65]  J. E. Rose,et al.  Some effects of stimulus intensity on response of auditory nerve fibers in the squirrel monkey. , 1971, Journal of neurophysiology.

[66]  A. Popper,et al.  The Evolutionary biology of hearing , 1992 .

[67]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[68]  L. T. Troland,et al.  The Principles of Psychophysiology , 1930 .

[69]  Ad Aertsen,et al.  From Synchrony to Harmony: Ideas on the Function of Neural Assemblies and on the Interpretation of Neural Synchrony , 1986 .

[70]  A. R. Palmer,et al.  Segregation of the Responses to Paired Vowels in the Auditory Nerve of the Guinea-Pig Using Autocorrelation , 1992 .

[71]  G. Laurent A systems perspective on early olfactory coding. , 1999, Science.

[72]  R Meddis,et al.  Modeling the identification of concurrent vowels with different fundamental frequencies. , 1992, The Journal of the Acoustical Society of America.

[73]  John G. Taylor,et al.  Self-organization in the time domain , 1998 .

[74]  Adrian Rees,et al.  Stimulus properties influencing the responses of inferior colliculus neurons to amplitude-modulated sounds , 1987, Hearing Research.

[75]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[76]  W. Reichardt,et al.  Autocorrelation, a principle for the evaluation of sensory information by the central nervous system , 1961 .

[77]  M. R. Jones,et al.  Time, our lost dimension: toward a new theory of perception, attention, and memory. , 1976, Psychological review.

[78]  W. Uttal The Psychobiology of Sensory Coding , 2014 .

[79]  Tatsuya Hirahara,et al.  REPRESENTATION OF LOW-FREQUENCY VOWEL FORMANTS IN THE AUDITORY NERVE , 2002 .

[80]  Rodney M. J. Cotterill Computer simulation in brain science: Contents , 1988 .

[81]  M. Abeles Role of the cortical neuron: integrator or coincidence detector? , 1982, Israel journal of medical sciences.

[82]  Gerald Langner,et al.  Periodicity coding in the auditory system , 1992, Hearing Research.

[83]  Q. Summerfield,et al.  Modeling the perception of concurrent vowels: vowels with the same fundamental frequency. , 1989, The Journal of the Acoustical Society of America.

[84]  Shihab Shamma,et al.  Auditory Representations of Timbre and Pitch , 1996 .

[85]  R. Eckhorn,et al.  Computer simulation in brain science: Texture description in the time domain , 1988 .

[86]  R. Reid,et al.  Temporal Coding of Visual Information in the Thalamus , 2000, The Journal of Neuroscience.

[87]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[88]  S. Grossberg,et al.  The Adaptive Brain , 1990 .

[89]  DeLiang Wang,et al.  An Oscillatory Correlation Theory of Temporal Pattern Segmentation , 1995 .

[90]  Axel Michelsen,et al.  Hearing and Sound Communication in Small Animals: Evolutionary Adaptations to the Laws of Physics , 1992 .

[91]  M. Sachs,et al.  Representation of steady-state vowels in the temporal aspects of the discharge patterns of populations of auditory-nerve fibers. , 1979, The Journal of the Acoustical Society of America.

[92]  Peter Cariani,et al.  Transient changes in neural discharge patterns may enhance the separation of concurrent vowels with different fundamental frequencies , 1994 .

[93]  G. Wilkinson The Theory of Hearing , 1925, Nature.

[94]  Alexander Joseph Book reviewDischarge patterns of single fibers in the cat's auditory nerve: Nelson Yuan-Sheng Kiang, with the assistance of Takeshi Watanabe, Eleanor C. Thomas and Louise F. Clark: Research Monograph no. 35. Cambridge, Mass., The M.I.T. Press, 1965 , 1967 .

[95]  Richard A. Young,et al.  Some observations on temporal coding of color vision: Psychophysical results , 1977, Vision Research.

[96]  S. Seneff A joint synchrony/mean-rate model of auditory speech processing , 1990 .

[97]  M. Semple,et al.  Auditory temporal processing: responses to sinusoidally amplitude-modulated tones in the inferior colliculus. , 2000, Journal of neurophysiology.

[98]  S Shamma,et al.  The case of the missing pitch templates: how harmonic templates emerge in the early auditory system. , 2000, The Journal of the Acoustical Society of America.

[99]  Koch Sigmund Ed,et al.  Psychology: A Study of A Science , 1962 .

[100]  E. de Boer,et al.  On the “Residue” and Auditory Pitch Perception , 1976 .

[101]  A. Cheveigné Cancellation model of pitch perception. , 1998 .

[102]  D. Amnon Silverstein,et al.  Vernier acuity during image rotation and translation: Visual performance limits , 1995, Vision Research.

[103]  S. A. Shamma The Auditory Processing of Speech. , 1986 .

[104]  Q. Summerfield,et al.  Modeling the perception of concurrent vowels: vowels with different fundamental frequencies. , 1990, The Journal of the Acoustical Society of America.

[105]  G. C. Quarton,et al.  The neurosciences : a study program , 1967 .

[106]  Richard F. Lyon,et al.  Computational models of neural auditory processing , 1984, ICASSP.

[107]  Pd Wall Pain: A Spike-Interval Coded Message in the Brain , 1982 .

[108]  B. Moore An Introduction to the Psychology of Hearing , 1977 .