Different timescales for the neural coding of consonant and vowel sounds.

Psychophysical, clinical, and imaging evidence suggests that consonant and vowel sounds have distinct neural representations. This study tests the hypothesis that consonant and vowel sounds are represented on different timescales within the same population of neurons by comparing behavioral discrimination with neural discrimination based on activity recorded in rat inferior colliculus and primary auditory cortex. Performance on 9 vowel discrimination tasks was highly correlated with neural discrimination based on spike count and was not correlated when spike timing was preserved. In contrast, performance on 11 consonant discrimination tasks was highly correlated with neural discrimination when spike timing was preserved and not when spike timing was eliminated. These results suggest that in the early stages of auditory processing, spike count encodes vowel sounds and spike timing encodes consonant sounds. These distinct coding strategies likely contribute to the robust nature of speech sound representations and may help explain some aspects of developmental and acquired speech processing disorders.

[1]  J. Mehler,et al.  Finding Words and Rules in a Speech Stream , 2008, Psychological science.

[2]  J. Rauschecker,et al.  Attention‐related modulation of activity in primary and secondary auditory cortex , 1997, Neuroreport.

[3]  A. Lotto,et al.  Neighboring spectral content influences vowel identification. , 2000, The Journal of the Acoustical Society of America.

[4]  Manuel Carreiras,et al.  Brain activation for consonants and vowels. , 2008, Cerebral cortex.

[5]  M. Sachs,et al.  Encoding of steady-state vowels in the auditory nerve: representation in terms of discharge rate. , 1979, The Journal of the Acoustical Society of America.

[6]  T Watanabe Responses of the cat's collicular auditory neuron to human speech. , 1978, The Journal of the Acoustical Society of America.

[7]  Sheila E Blumstein,et al.  Temporal integration in vowel perception. , 2009, The Journal of the Acoustical Society of America.

[8]  Megan M. Kittleson,et al.  Cortical and subcortical contributions to the attentive processing of speech , 2008, Neuroreport.

[9]  G. Recanzone,et al.  Serial and parallel processing in the primate auditory cortex revisited , 2010, Behavioural Brain Research.

[10]  Brian J Malone,et al.  Temporal Codes for Amplitude Contrast in Auditory Cortex , 2010, The Journal of Neuroscience.

[11]  Eric D. Young Level and spectrum , 2010 .

[12]  A. Caramazza,et al.  Separable processing of consonants and vowels , 2000, Nature.

[13]  Xiaoqin Wang,et al.  Sustained firing in auditory cortex evoked by preferred stimuli , 2005, Nature.

[14]  H. Vaughan,et al.  Tonotopic features of speech-evoked activity in primate auditory cortex , 1990, Brain Research.

[15]  Nicolas Brunel,et al.  Sensory neural codes using multiplexed temporal scales , 2010, Trends in Neurosciences.

[16]  W. Maass,et al.  State-dependent computations: spatiotemporal processing in cortical networks , 2009, Nature Reviews Neuroscience.

[17]  J. Rauschecker,et al.  Vowel sound extraction in anterior superior temporal cortex , 2006, Human brain mapping.

[18]  D. Sinex,et al.  Neural responses to the onset of voicing are unrelated to other measures of temporal resolution. , 1997, The Journal of the Acoustical Society of America.

[19]  J. Rauschecker,et al.  Mechanisms and streams for processing of "what" and "where" in auditory cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Kerry M. M. Walker,et al.  Multiplexed and Robust Representations of Sound Features in Auditory Cortex , 2011, The Journal of Neuroscience.

[21]  Bryan E Pfingst,et al.  Relative contributions of spectral and temporal cues for phoneme recognition. , 2005, The Journal of the Acoustical Society of America.

[22]  L H Carney,et al.  A temporal analysis of auditory-nerve fiber responses to spoken stop consonant-vowel syllables. , 1986, The Journal of the Acoustical Society of America.

[23]  W. Strange,et al.  Dynamic specification of coarticulated vowels spoken in sentence context. , 1989, The Journal of the Acoustical Society of America.

[24]  Emilio Salinas,et al.  Cognitive neuroscience: Flutter Discrimination: neural codes, perception, memory and decision making , 2003, Nature Reviews Neuroscience.

[25]  S. Shamma,et al.  Spectral-ripple representation of steady-state vowels in primary auditory cortex. , 1998, The Journal of the Acoustical Society of America.

[26]  Michael N. Shadlen,et al.  Noise, neural codes and cortical organization , 1994, Current Opinion in Neurobiology.

[27]  A. Liberman,et al.  The Identification and Discrimination of Synthetic Vowels , 1962 .

[28]  Alan R. Palmer,et al.  The Oxford Handbook of Auditory Science: The Auditory Brain , 2010 .

[29]  M. Kilgard,et al.  Cortical activity patterns predict robust speech discrimination ability in noise , 2011, The European journal of neuroscience.

[30]  G. A. Miller,et al.  An Analysis of Perceptual Confusions Among Some English Consonants , 1955 .

[31]  T. Hromádka,et al.  Reliability and Representational Bandwidth in the Auditory Cortex , 2005, Neuron.

[32]  Mitchell Steinschneider,et al.  Speech evoked activity in the auditory radiations and cortex of the awake monkey , 1982, Brain Research.

[33]  M Steinschneider,et al.  Temporal encoding of the voice onset time phonetic parameter by field potentials recorded directly from human auditory cortex. , 1999, Journal of neurophysiology.

[34]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[35]  P. C. Murphy,et al.  Cerebral Cortex , 2017, Cerebral Cortex.

[36]  Ranulfo Romo,et al.  Neural codes for perceptual discrimination of acoustic flutter in the primate auditory cortex , 2009, Proceedings of the National Academy of Sciences.

[37]  B. Richmond,et al.  Latency: another potential code for feature binding in striate cortex. , 1996, Journal of neurophysiology.

[38]  M. Kilgard,et al.  Discrimination of brief speech sounds is impaired in rats with auditory cortex lesions , 2011, Behavioural Brain Research.

[39]  C D Geisler,et al.  Responses of auditory-nerve fibers to nasal consonant-vowel syllables. , 1987, The Journal of the Acoustical Society of America.

[40]  K. Hugdahl,et al.  The effects of attention on speech perception: An fMRI study , 2003, Brain and Language.

[41]  H Scheich,et al.  Orderly cortical representation of vowels based on formant interaction. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[42]  S. Soli Second formants in fricatives: Acoustic consequences of fricative‐vowel coarticulation , 1981 .

[43]  David Poeppel,et al.  The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time' , 2003, Speech Commun..

[44]  R. Lesser,et al.  Auditory perception of segmental features: a functional-neuroanatomic study , 1994, Journal of Neurolinguistics.

[45]  A. Liberman,et al.  An Experimental Study of the Acoustic Determinants of Vowel Color; Observations on One- and Two-Formant Vowels Synthesized from Spectrographic Patterns , 1952 .

[46]  Y. Cohen,et al.  Representation of speech categories in the primate auditory cortex. , 2011, Journal of neurophysiology.

[47]  W. Newsome,et al.  Correlation between Speed Perception and Neural Activity in the Middle Temporal Visual Area , 2005, The Journal of Neuroscience.

[48]  Nima Mesgarani,et al.  Phoneme representation and classification in primary auditory cortex. , 2008, The Journal of the Acoustical Society of America.

[49]  J. Werker,et al.  Developmental changes in perception of nonnative vowel contrasts. , 1994, Journal of experimental psychology. Human perception and performance.

[50]  R. Lesser,et al.  Auditory Speech Processing in the Left Temporal Lobe: An Electrical Interference Study , 1995, Brain and Language.

[51]  G. Foffani,et al.  Spike Timing, Spike Count, and Temporal Information for the Discrimination of Tactile Stimuli in the Rat Ventrobasal Complex , 2009, The Journal of Neuroscience.

[52]  Jan W. H. Schnupp,et al.  Plasticity of Temporal Pattern Codes for Vocalization Stimuli in Primary Auditory Cortex , 2006, The Journal of Neuroscience.

[53]  H. Burton,et al.  Neural correlates for roughness choice in monkey second somatosensory cortex (SII). , 2001, Journal of neurophysiology.

[54]  D. M. Green,et al.  A panoramic code for sound location by cortical neurons. , 1994, Science.

[55]  M. Sachs,et al.  Representation of steady-state vowels in the temporal aspects of the discharge patterns of populations of auditory-nerve fibers. , 1979, The Journal of the Acoustical Society of America.

[56]  R. Zatorre,et al.  Structure and function of auditory cortex: music and speech , 2002, Trends in Cognitive Sciences.

[57]  M. Kilgard,et al.  Anesthesia suppresses nonsynchronous responses to repetitive broadband stimuli , 2007, Neuroscience.

[58]  R V Shannon,et al.  Speech Recognition with Primarily Temporal Cues , 1995, Science.

[59]  A. Boemio,et al.  Hierarchical and asymmetric temporal sensitivity in human auditory cortices , 2005, Nature Neuroscience.

[60]  Jonathan D. Victor,et al.  How the brain uses time to represent and process visual information 1 1 Published on the World Wide Web on 16 August 2000. , 2000, Brain Research.

[61]  K. Shibuki,et al.  Requirement of the auditory association cortex for discrimination of vowel-like sounds in rats , 2006, Neuroreport.

[62]  Hideki Kawahara,et al.  Speech representation and transformation using adaptive interpolation of weighted spectrum: vocoder revisited , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[63]  D. Caspary,et al.  Physiology of the young adult Fischer 344 rat inferior colliculus: responses to contralateral monaural stimuli , 1996, Hearing Research.

[64]  Mitchell Steinschneider,et al.  Intracortical responses in human and monkey primary auditory cortex support a temporal processing mechanism for encoding of the voice onset time phonetic parameter. , 2004, Cerebral cortex.

[65]  John G. Neuhoff,et al.  Spatiotemporal Pattern of Neural Processing in the Human Auditory Cortex , 2002, Science.

[66]  Christoph E. Schreiner,et al.  Representation of CV-sounds in cat primary auditory cortex: intensity dependence , 2003, Speech Commun..

[67]  P. Tallal,et al.  Rate of acoustic change may underlie hemispheric specialization for speech perception , 1980, Science.

[68]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[69]  A. Parker,et al.  Sense and the single neuron: probing the physiology of perception. , 1998, Annual review of neuroscience.

[70]  J. Mehler,et al.  Linguistic Constraints on Statistical Computations , 2005, Psychological science.

[71]  Eaton Peabody,et al.  As If Time Really Mattered: Temporal Strategies for Neural Coding of Sensory Information , 2018, Origins.

[72]  J. Eggermont Representation of a voice onset time continuum in primary auditory cortex of the cat. , 1995, The Journal of the Acoustical Society of America.

[73]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[74]  Nina Kraus,et al.  Effects of noise and cue enhancement on neural responses to speech in auditory midbrain, thalamus and cortex , 2002, Hearing Research.

[75]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[76]  R. Romo,et al.  Decoding a Perceptual Decision Process across Cortex , 2010, Neuron.

[77]  M. Goldstein,et al.  Neuroperceptual Differences in Consonant and Vowel Discrimination: As Revealed by Direct Cortical Electrical Interference , 1997, Cortex.

[78]  S. Scott,et al.  The neuroanatomical and functional organization of speech perception , 2003, Trends in Neurosciences.

[79]  Guglielmo Foffani,et al.  PSTH-based classification of sensory stimuli using ensembles of single neurons , 2004, Journal of Neuroscience Methods.

[80]  B. Delgutte,et al.  Speech coding in the auditory nerve: III. Voiceless fricative consonants. , 1984, The Journal of the Acoustical Society of America.

[81]  M. Kilgard,et al.  Cortical activity patterns predict speech discrimination ability , 2008, Nature Neuroscience.

[82]  G. E. Peterson,et al.  Control Methods Used in a Study of the Vowels , 1951 .

[83]  Gal Chechik,et al.  Reduction of Information Redundancy in the Ascending Auditory Pathway , 2006, Neuron.

[84]  B. Delgutte,et al.  Speech coding in the auditory nerve: I. Vowel-like sounds. , 1984, The Journal of the Acoustical Society of America.

[85]  M M Merzenich,et al.  Representation of a species-specific vocalization in the primary auditory cortex of the common marmoset: temporal and spectral characteristics. , 1995, Journal of neurophysiology.

[86]  Philipos C Loizou,et al.  The intelligibility of speech with "holes" in the spectrum. , 2002, The Journal of the Acoustical Society of America.

[87]  C. Schroeder,et al.  Tonotopic organization of responses reflecting stop consonant place of articulation in primary auditory cortex (A1) of the monkey , 1995, Brain Research.

[88]  M. Sachs,et al.  Representation of stop consonants in the discharge patterns of auditory-nerve fibers. , 1983, The Journal of the Acoustical Society of America.

[89]  B. Auditory and phonetic memory codes in the discrimination of consonants and vowels * , 2022 .

[90]  S. Lomber,et al.  Double dissociation of 'what' and 'where' processing in auditory cortex , 2008, Nature Neuroscience.

[91]  R. L. Rennaker,et al.  Response to broadband repetitive stimuli in auditory cortex of the unanesthetized rat , 2006, Hearing Research.

[92]  M. Studdert-Kennedy,et al.  Identification of Consonants and Vowels Presented to Left and Right Ears* , 1967, The Quarterly journal of experimental psychology.

[93]  P. Chauvel,et al.  Specialization of left auditory cortex for speech perception in man depends on temporal coding. , 1999, Cerebral cortex.

[94]  A R Palmer,et al.  The representation of the spectra and fundamental frequencies of steady-state single- and double-vowel sounds in the temporal discharge patterns of guinea pig cochlear-nerve fibers. , 1990, The Journal of the Acoustical Society of America.

[95]  M. E. Raichle,et al.  PET Studies of Auditory and Phonological Processing: Effects of Stimulus Characteristics and Task Demands , 1995, Journal of Cognitive Neuroscience.

[96]  P. Latham,et al.  Ruling out and ruling in neural codes , 2009, Proceedings of the National Academy of Sciences.