Exploring the Roles of Spectral Detail and Intonation Contour in Speech Intelligibility: An fMRI Study

The melodic contour of speech forms an important perceptual aspect of tonal and nontonal languages and an important limiting factor on the intelligibility of speech heard through a cochlear implant. Previous work exploring the neural correlates of speech comprehension identified a left-dominant pathway in the temporal lobes supporting the extraction of an intelligible linguistic message, whereas the right anterior temporal lobe showed an overall preference for signals clearly conveying dynamic pitch information [Johnsrude, I. S., Penhune, V. B., & Zatorre, R. J. Functional specificity in the right human auditory cortex for perceiving pitch direction. Brain, 123, 155–163, 2000; Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. Identification of a pathway for intelligible speech in the left temporal lobe. Brain, 123, 2400–2406, 2000]. The current study combined modulations of overall intelligibility (through vocoding and spectral inversion) with a manipulation of pitch contour (normal vs. falling) to investigate the processing of spoken sentences in functional MRI. Our overall findings replicate and extend those of Scott et al. [Scott, S. K., Blank, C. C., Rosen, S., & Wise, R. J. Identification of a pathway for intelligible speech in the left temporal lobe. Brain, 123, 2400–2406, 2000], where greater sentence intelligibility was predominately associated with increased activity in the left STS, and the greatest response to normal sentence melody was found in right superior temporal gyrus. These data suggest a spatial distinction between brain areas associated with intelligibility and those involved in the processing of dynamic pitch information in speech. By including a set of complexity-matched unintelligible conditions created by spectral inversion, this is additionally the first study reporting a fully factorial exploration of spectrotemporal complexity and spectral inversion as they relate to the neural processing of speech intelligibility. Perhaps surprisingly, there was little evidence for an interaction between the two factors—we discuss the implications for the processing of sound and speech in the dorsolateral temporal lobes.

[1]  Martin Walger,et al.  The perception of prosody and speaker gender in normal-hearing listeners and cochlear implant recipients , 2009, International journal of audiology.

[2]  John F Culling,et al.  The role of fundamental frequency contours in the perception of speech against interfering speech. , 2005, The Journal of the Acoustical Society of America.

[3]  Robert S Schlauch,et al.  Fundamental frequency variation with an electrolarynx improves speech understanding: a case study. , 2009, American journal of speech-language pathology.

[4]  Jonathan Hutchinson,et al.  What? When? And How? , 1890, The Hospital.

[5]  Matthew H. Davis,et al.  Predictive Top-Down Integration of Prior Knowledge during Speech Perception , 2012, The Journal of Neuroscience.

[6]  J Bamford,et al.  The BKB (Bamford-Kowal-Bench) sentence lists for partially-hearing children. , 1979, British journal of audiology.

[7]  Matthew H. Davis,et al.  Hierarchical Processing in Spoken Language Comprehension , 2003, The Journal of Neuroscience.

[8]  Jonathan H. Venezia,et al.  Hierarchical organization of human auditory cortex: evidence from acoustic invariance in the response to intelligible speech. , 2010, Cerebral cortex.

[9]  Jay T Rubinstein,et al.  How cochlear implants encode speech , 2004, Current opinion in otolaryngology & head and neck surgery.

[10]  S. Nooteboom,et al.  THE PROSODY OF SPEECH: MELODY AND RHYTHM , 2001 .

[11]  S. Scott,et al.  The Pathways for Intelligible Speech: Multivariate and Univariate Perspectives , 2013, Cerebral cortex.

[12]  Angela D. Friederici,et al.  Hemispheric lateralization of linguistic prosody recognition in comparison to speech and speaker recognition , 2014, NeuroImage.

[13]  Sophie K. Scott,et al.  Hemispheric Asymmetries in Speech Perception: Sense, Nonsense and Modulations , 2011, PloS one.

[14]  A. Kleinschmidt,et al.  Modulation of neural responses to speech by directing attention to voices or verbal content. , 2003, Brain research. Cognitive brain research.

[15]  Sophie K. Scott,et al.  An Application of Univariate and Multivariate Approaches in fMRI to Quantifying the Hemispheric Lateralization of Acoustic and Linguistic Processes , 2012, Journal of Cognitive Neuroscience.

[16]  S. Scott,et al.  The neuroanatomical and functional organization of speech perception , 2003, Trends in Neurosciences.

[17]  P. Matthews,et al.  Defining a left-lateralized response specific to intelligible speech using fMRI. , 2003, Cerebral cortex.

[18]  R. Zatorre,et al.  Functional specificity in the right human auditory cortex for perceiving pitch direction. , 2000, Brain : a journal of neurology.

[19]  Martin Walger,et al.  Use of intonation contours for speech recognition in noise by cochlear implant recipients. , 2011, The Journal of the Acoustical Society of America.

[20]  M. Pell,et al.  The neural bases of prosody: Insights from lesion studies and neuroimaging , 1999 .

[21]  R. Zatorre,et al.  ‘What’, ‘where’ and ‘how’ in auditory cortex , 2000, Nature Neuroscience.

[22]  Mirjam Ernestus,et al.  An unfamiliar intonation contour slows down online speech comprehension , 2011 .

[23]  Hideki Kawahara,et al.  Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds , 1999, Speech Commun..

[24]  Daniela Sammler,et al.  Prosody meets syntax: the role of the corpus callosum. , 2010, Brain : a journal of neurology.

[25]  Tom Manly,et al.  T'ain't What You Say, It's the Way That You Say It—Left Insula and Inferior Frontal Cortex Work in Interaction with Superior Temporal Regions to Control the Performance of Vocal Impersonations , 2013, Journal of Cognitive Neuroscience.

[26]  Joachim Gross,et al.  Phase-Locked Responses to Speech in Human Auditory Cortex are Enhanced During Comprehension , 2012, Cerebral cortex.

[27]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[28]  Pascal Belin,et al.  Right temporal TMS impairs voice detection , 2011, Current Biology.

[29]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[30]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[31]  Stuart Rosen,et al.  Neural correlates of intelligibility in speech investigated with noise vocoded speech--a positron emission tomography study. , 2006, The Journal of the Acoustical Society of America.

[32]  D. T. Ives,et al.  Bihemispheric foundations for human speech comprehension , 2010, Proceedings of the National Academy of Sciences.

[33]  Sophie K. Scott,et al.  Cortical asymmetries in speech perception: what's wrong, what's right and what's left? , 2012, Trends in Cognitive Sciences.

[34]  Mario Dzemidzic,et al.  Hemispheric roles in the perception of speech prosody , 2004, NeuroImage.

[35]  R. Wise,et al.  Sounds do-able: auditory–motor transformations and the posterior temporal plane , 2005, Trends in Neurosciences.

[36]  David Poeppel,et al.  The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time' , 2003, Speech Commun..

[37]  Mohamed L. Seghier,et al.  Laterality index in functional MRI: methodological issues☆ , 2008, Magnetic resonance imaging.

[38]  R. Zatorre,et al.  Spectral and temporal processing in human auditory cortex. , 2001, Cerebral cortex.

[39]  R. Zatorre,et al.  Human temporal-lobe response to vocal sounds. , 2002, Brain research. Cognitive brain research.

[40]  Fan-Gang Zeng,et al.  Fundamental frequency is critical to speech perception in noise in combined acoustic and electric hearing. , 2011, The Journal of the Acoustical Society of America.

[41]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[42]  S. Scott,et al.  Speech comprehension aided by multiple modalities: Behavioural and neural interactions , 2012, Neuropsychologia.

[43]  R. Bowtell,et al.  “sparse” temporal sampling in auditory fMRI , 1999, Human brain mapping.

[44]  D. D. Greenwood.  A cochlear frequency-position function for several species--29 years later. , 1990, The Journal of the Acoustical Society of America.

[45]  P. Wagner,et al.  The Perception of Sentence Stress in Cochlear Implant Recipients , 2011, Ear and hearing.

[46]  S. Scott,et al.  Inferior Frontal Gyrus Activation Predicts Individual Differences in Perceptual Learning of Cochlear-Implant Simulations , 2010, The Journal of Neuroscience.

[47]  Andreas Kleinschmidt,et al.  Interaction of Face and Voice Areas during Speaker Recognition , 2005, Journal of Cognitive Neuroscience.

[48]  D. V. von Cramon,et al.  FMRI reveals brain regions mediating slow prosodic modulations in spoken sentences , 2002, Human brain mapping.

[49]  A. Botinis,et al.  Intonation , 2001, Speech Commun..

[50]  Vincent Schmithorst,et al.  A combined bootstrap/histogram analysis approach for computing a lateralization index from neuroimaging data , 2006, NeuroImage.

[51]  Ann Cutler,et al.  Prosody in the Comprehension of Spoken Language: A Literature Review , 1997, Language and speech.

[52]  Stuart Rosen,et al.  Enhancing temporal cues to voice pitch in continuous interleaved sampling cochlear implants. , 2004, The Journal of the Acoustical Society of America.

[53]  Robert S Schlauch,et al.  The effects of fundamental frequency contour manipulations on speech intelligibility in background noise. , 2010, The Journal of the Acoustical Society of America.

[54]  R. Zatorre,et al.  Musical Melody and Speech Intonation: Singing a Different Tune , 2012, PLoS biology.

[55]  Christopher Turner,et al.  Accuracy of Cochlear Implant Recipients on Pitch Perception, Melody Recognition, and Speech Reception in Noise , 2007, Ear and hearing.

[56]  R. Weisskoff,et al.  Improved auditory cortex imaging using clustered volume acquisitions , 1999, Human brain mapping.

[57]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[58]  R. Zatorre,et al.  Adaptation to speaker's voice in right anterior temporal lobe , 2003, Neuroreport.

[59]  Anne-Lise Giraud,et al.  Distinct functional substrates along the right superior temporal sulcus for the processing of voices , 2004, NeuroImage.

[60]  Mario Dzemidzic,et al.  Neural circuitry underlying sentence-level linguistic prosody , 2005, NeuroImage.

[61]  Bei Wang,et al.  A music perception disorder (congenital amusia) influences speech comprehension , 2015, Neuropsychologia.

[62]  S. Trehub,et al.  Effect of cochlear implants on children's perception and production of speech prosody. , 2012, The Journal of the Acoustical Society of America.

[63]  J. Hart,et al.  Distinct prefrontal cortex activity associated with item memory and source memory for visual shapes. , 2003, Brain research. Cognitive brain research.