Phonetic Feature Encoding in Human Superior Temporal Gyrus

Deciphering Language Consonants and vowels represent basic building blocks of human language. How their characteristics are extracted from acoustic speech input is not well understood. Directly recording from the superior temporal gyrus of patients as part of their clinical evaluation for epilepsy surgery, Mesgarani et al. (p. 1006, published online 30 January; see the Perspective by Grodzinsky and Nelken) investigated neural responses while the subjects listened to continuous speech. The findings reveal how both vowels and consonants of different phonetic categories are encoded. The human auditory cortex encodes what speech sounds like. [Also see Perspective by Grodzinsky and Nelken] During speech perception, linguistic elements such as consonants and vowels are extracted from a complex acoustic speech signal. The superior temporal gyrus (STG) participates in high-order auditory processing of speech, but how it encodes phonetic information is poorly understood. We used high-density direct cortical surface recordings in humans while they listened to natural, continuous speech to reveal the STG representation of the entire English phonetic inventory. At single electrodes, we found response selectivity to distinct phonetic features. Encoding of acoustic properties was mediated by a distributed population response. Phonetic features could be directly related to tuning for spectrotemporal acoustic cues, some of which were encoded in a nonlinear fashion or by integration of multiple cues. These findings demonstrate the acoustic-phonetic representation of speech in human STG.

[1]  Carla Teixeira Lopes,et al.  TIMIT Acoustic-Phonetic Continuous Speech Corpus , 2012 .

[2]  M. Kilgard,et al.  Cortical activity patterns predict speech discrimination ability , 2008, Nature Neuroscience.

[3]  H. S. Gopal,et al.  A perceptual model of vowel recognition based on the auditory representation of American English vowels. , 1986, The Journal of the Acoustical Society of America.

[4]  Mitchell Steinschneider,et al.  Intracortical responses in human and monkey primary auditory cortex support a temporal processing mechanism for encoding of the voice onset time phonetic parameter. , 2004, Cerebral cortex.

[5]  J. Maunsell,et al.  Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex , 2011, PLoS biology.

[6]  N. C. Singh,et al.  Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli , 2001 .

[7]  M. Goldstein,et al.  Neuroperceptual Differences in Consonant and Vowel Discrimination: As Revealed by Direct Cortical Electrical Interference , 1997, Cortex.

[8]  Mitchell Steinschneider,et al.  Spectrotemporal analysis of evoked and induced electroencephalographic responses in primary auditory cortex (A1) of the awake monkey. , 2008, Cerebral cortex.

[9]  Vinay Jayaram,et al.  Speech-specific tuning of neurons in human superior temporal gyrus. , 2014, Cerebral cortex.

[10]  O. Creutzfeldt,et al.  Neuronal activity in the human lateral temporal lobe , 1989, Experimental Brain Research.

[11]  Mitchell Steinschneider,et al.  Intracranial study of speech-elicited activity on the human posterolateral superior temporal gyrus. , 2011, Cerebral cortex.

[12]  宁北芳,et al.  疟原虫var基因转换速率变化导致抗原变异[英]/Paul H, Robert P, Christodoulou Z, et al//Proc Natl Acad Sci U S A , 2005 .

[13]  G. A. Miller,et al.  An Analysis of Perceptual Confusions Among Some English Consonants , 1955 .

[14]  Harvey M. Sussman,et al.  A neuronal model of vowel normalization and representation , 1986, Brain and Language.

[15]  O. Creutzfeldt,et al.  Neuronal activity in the human lateral temporal lobe , 2004, Experimental Brain Research.

[16]  James D. Miller Auditory‐perceptual interpretation of the vowel , 1987 .

[17]  Gal Chechik,et al.  Auditory abstraction from spectro-temporal features to coding auditory entities , 2012, Proceedings of the National Academy of Sciences.

[18]  J. Rauschecker,et al.  Segregation of Vowels and Consonants in Human Auditory Cortex: Evidence for Distributed Hierarchical Organization , 2010, Front. Psychology.

[19]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[20]  Kenneth N Stevens,et al.  Toward a model for lexical access based on acoustic landmarks and distinctive features. , 2002, The Journal of the Acoustical Society of America.

[21]  Kuansan Wang,et al.  Self-normalization and noise-robustness in early auditory representations , 1994, IEEE Trans. Speech Audio Process..

[22]  P. Ladefoged A course in phonetics , 1975 .

[23]  L. Lisker,et al.  Some Effects of Context On Voice Onset Time in English Stops , 1967, Language and speech.

[24]  G. E. Peterson,et al.  Control Methods Used in a Study of the Vowels , 1951 .

[25]  J. van Loon Network , 2006 .

[26]  George N. Clements,et al.  The geometry of phonological features , 1985, Phonology Yearbook.

[27]  J. P. Lee,et al.  Wenner-Gren International Symposium Series , 1990 .

[28]  Jonathan G. Fiscus,et al.  Darpa Timit Acoustic-Phonetic Continuous Speech Corpus CD-ROM {TIMIT} | NIST , 1993 .

[29]  I. Nelken,et al.  Processing of complex sounds in the auditory system , 2008, Current Opinion in Neurobiology.

[30]  K. Stevens,et al.  Knowledge of language and the sounds of speech , 1991 .

[31]  B. Gordon,et al.  Induced electrocorticographic gamma activity during auditory perception , 2001, Clinical Neurophysiology.

[32]  E. Chang,et al.  Categorical Speech Representation in Human Superior Temporal Gyrus , 2010, Nature Neuroscience.

[33]  Noam Chomsky,et al.  The Sound Pattern of English , 1968 .

[34]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[35]  Alvin M. Liberman,et al.  Speech: A Special Code , 1996 .

[36]  C. Fowler An event approach to the study of speech perception from a direct realist perspective , 1986 .

[37]  Erik Edwards,et al.  Comparison of time-frequency responses and the event-related potential to auditory speech stimuli in human cortex. , 2009, Journal of neurophysiology.

[38]  Nima Mesgarani,et al.  Phoneme representation and classification in primary auditory cortex. , 2008, The Journal of the Acoustical Society of America.