Right Posterior Temporal Cortex Supports Integration of Phonetic and Talker Information

Abstract Though the right hemisphere has been implicated in talker processing, it is thought to play a minimal role in phonetic processing, at least relative to the left hemisphere. Recent evidence suggests that the right posterior temporal cortex may support learning of phonetic variation associated with a specific talker. In the current study, listeners heard a male talker and a female talker, one of whom produced an ambiguous fricative in /s/-biased lexical contexts (e.g., epi?ode) and one who produced it in /∫/-biased contexts (e.g., friend?ip). Listeners in a behavioral experiment (Experiment 1) showed evidence of lexically guided perceptual learning, categorizing ambiguous fricatives in line with their previous experience. Listeners in an fMRI experiment (Experiment 2) showed differential phonetic categorization as a function of talker, allowing for an investigation of the neural basis of talker-specific phonetic processing, though they did not exhibit perceptual learning (likely due to characteristics of our in-scanner headphones). Searchlight analyses revealed that the patterns of activation in the right superior temporal sulcus (STS) contained information about who was talking and what phoneme they produced. We take this as evidence that talker information and phonetic information are integrated in the right STS. Functional connectivity analyses suggested that the process of conditioning phonetic identity on talker information depends on the coordinated activity of a left-lateralized phonetic processing system and a right-lateralized talker processing system. Overall, these results clarify the mechanisms through which the right hemisphere supports talker-specific phonetic processing.

[1]  Emily B. Myers,et al.  Perceptual learning of multiple talkers requires additional exposure , 2021, Attention, Perception, & Psychophysics.

[2]  Emily B. Myers,et al.  Listeners are initially flexible in updating phonetic beliefs over time , 2021, Psychonomic Bulletin & Review.

[3]  Christina Y. Tzeng,et al.  A second chance for a first impression: Sensitivity to cumulative input statistics for lexically guided perceptual learning , 2021, Psychonomic Bulletin & Review.

[4]  Sahil Luthra The Role of the Right Hemisphere in Processing Phonetic Variability Between Talkers , 2020, Neurobiology of Language.

[5]  Anne Marie Crinnion,et al.  A graph-theoretic approach to identifying acoustic cues for speech sound categorization , 2020, Psychonomic Bulletin & Review.

[6]  Emily B. Myers,et al.  Lexical Information Guides Retuning of Neural Patterns in Perceptual Learning for Speech , 2020, Journal of Cognitive Neuroscience.

[7]  A. Jesse Sentence context guides phonetic retuning to speaker idiosyncrasies. , 2019, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Efthymia C. Kapnoula,et al.  Voices in the mental lexicon: Words carry indexical information that can affect access to their meaning , 2019, Journal of Memory and Language.

[9]  Matthew K. Leonard,et al.  The Encoding of Speech Sounds in the Superior Temporal Gyrus , 2019, Neuron.

[10]  Emily B. Myers,et al.  Neural substrates of subphonemic variation and lexical competition in spoken word recognition , 2018, Language, cognition and neuroscience.

[11]  Natasha Z. Kirkham,et al.  Gorilla in our midst: An online behavioral experiment builder , 2018, Behavior Research Methods.

[12]  Rachel M. Theodore,et al.  Lexically guided perceptual learning is robust to task-based changes in listening strategy. , 2018, The Journal of the Acoustical Society of America.

[13]  Dave F. Kleinschmidt,et al.  Structure in talker variability: How much is there and how much can it help? , 2018, Language, cognition and neuroscience.

[14]  Corrina Maguinness,et al.  Understanding the mechanisms of familiar voice-identity recognition in the human brain , 2018, Neuropsychologia.

[15]  Neal P. Fox,et al.  Speaker information affects false recognition of unstudied lexical-semantic associates , 2018, Attention, Perception, & Psychophysics.

[16]  Xin Xie,et al.  Left Inferior Frontal Gyrus Sensitivity to Phonetic Competition in Receptive Language Processing: A Comparison of Clear and Conversational Speech , 2018, Journal of Cognitive Neuroscience.

[17]  Chris I. Baker,et al.  Deconstructing multivariate decoding for the study of brain function , 2017, NeuroImage.

[18]  M. Silvestrini,et al.  Selective associative phonagnosia after right anterior temporal stroke , 2017, Neuropsychologia.

[19]  Josh H McDermott,et al.  Headphone screening to facilitate web-based auditory experiments , 2017, Attention, Perception, & Psychophysics.

[20]  Matthew H. Davis,et al.  Inferior Frontal Cortex Contributions to the Recognition of Spoken Words and Their Constituent Speech Sounds , 2017, Journal of Cognitive Neuroscience.

[21]  Emily B. Myers,et al.  Voice-sensitive brain networks encode talker-specific phonetic detail , 2017, Brain and Language.

[22]  Milene Bonte,et al.  Decoding Articulatory Features from fMRI Responses in Dorsal Speech Regions , 2015, The Journal of Neuroscience.

[23]  D. Bates,et al.  Balancing Type I Error and Power in Linear Mixed Models , 2015, 1511.01864.

[24]  Pascal Belin,et al.  Hemispheric association and dissociation of voice and speech information processing in stroke , 2015, Cortex.

[25]  Matthew H. Davis,et al.  Hierarchical Organization of Auditory and Motor Representations in Speech Perception: Evidence from Searchlight Similarity Analysis , 2015, Cerebral cortex.

[26]  Rachel M. Theodore,et al.  Attention modulates specificity effects in spoken word recognition: Challenges to the time-course hypothesis , 2015, Attention, perception & psychophysics.

[27]  Jessica S. Arsenault,et al.  Distributed Neural Representations of Phonological Features during Speech Perception , 2015, The Journal of Neuroscience.

[28]  Martin N. Hebart,et al.  The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data , 2015, Front. Neuroinform..

[29]  Stefan J. Kiebel,et al.  Voice Identity Recognition: Functional Division of the Right STS and Its Behavioral Relevance , 2014, Journal of Cognitive Neuroscience.

[30]  Emily B. Myers,et al.  Neural Systems Underlying Perceptual Adjustment to Non-Standard Speech Tokens. , 2014, Journal of memory and language.

[31]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[32]  Keith Johnson,et al.  Phonetic Feature Encoding in Human Superior Temporal Gyrus , 2014, Science.

[33]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[34]  James M. McQueen,et al.  Mean-based neural coding of voices , 2013, NeuroImage.

[35]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[36]  Bruce Fischl,et al.  FreeSurfer , 2012, NeuroImage.

[37]  Sophie K. Scott,et al.  Cortical asymmetries in speech perception: what's wrong, what's right and what's left? , 2012, Trends in Cognitive Sciences.

[38]  Richard Granger,et al.  Categorical Speech Processing in Broca's Area: An fMRI Study Using Multivariate Pattern-Based Analysis , 2012, The Journal of Neuroscience.

[39]  Russell A. Poldrack,et al.  Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses , 2012, NeuroImage.

[40]  Jan Theeuwes,et al.  OpenSesame: An open-source, graphical experiment builder for the social sciences , 2011, Behavior Research Methods.

[41]  Ferath Kherif,et al.  Does Semantic Context Benefit Speech Understanding through “Top–Down” Processes? Evidence from Time-resolved Sparse fMRI , 2011, Journal of Cognitive Neuroscience.

[42]  Pascal Belin,et al.  Right temporal TMS impairs voice detection , 2011, Current Biology.

[43]  J. McQueen,et al.  Positional effects in the lexical retuning of speech perception , 2011, Psychonomic bulletin & review.

[44]  B. McMurray,et al.  What information is necessary for speech categorization? Harnessing variability in the speech signal by integrating cues computed relative to expectations. , 2011, Psychological review.

[45]  Katherine S White,et al.  Adaptation to novel accents by toddlers. , 2011, Developmental science.

[46]  E. Formisano,et al.  Auditory Cortex Encodes the Perceptual Interpretation of Ambiguous Sound , 2011, The Journal of Neuroscience.

[47]  H. Coslett,et al.  Localization of sublexical speech perception components , 2010, Brain and Language.

[48]  Rutvik H. Desai,et al.  Specialization along the Left Superior Temporal Sulcus for Auditory Categorization , 2010, Cerebral cortex.

[49]  Stefan J Kiebel,et al.  How the Human Brain Recognizes Speech in the Context of Changing Speakers , 2010, The Journal of Neuroscience.

[50]  Rachel M. Theodore,et al.  Characteristics of listener sensitivity to talker-specific phonetic detail. , 2008, The Journal of the Acoustical Society of America.

[51]  A. Samuel,et al.  Perceptual learning for speech , 2009, Attention, perception & psychophysics.

[52]  Emily B. Myers,et al.  Inferior Frontal Regions Underlie the Perception of Phonetic Category Invariance , 2009, Psychological science.

[53]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[54]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[55]  Rainer Goebel,et al.  Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns , 2008, NeuroImage.

[56]  Sven L Mattys,et al.  On building models of spoken-word recognition: When there is as much to learn from natural “oddities” as artificial normality , 2008, Perception & psychophysics.

[57]  Jeffrey R. Binder,et al.  Left Posterior Temporal Regions are Sensitive to Auditory Categorization , 2008, Journal of Cognitive Neuroscience.

[58]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[59]  Hideki Kawahara,et al.  Tandem-STRAIGHT: A temporally stable power spectral representation for periodic signals and applications to interference-free spectrum, F0, and aperiodicity estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[60]  Michael K. Tanenhaus,et al.  The Weckud Wetch of the Wast: Lexical Adaptation to a Novel Accent , 2008, Cogn. Sci..

[61]  Matthias J. Sjerps,et al.  Speaker Normalization in Speech Perception , 2008, The Handbook of Speech Perception.

[62]  Emily B. Myers Dissociable effects of phonetic competition and category typicality in a phonetic categorization task: An fMRI investigation , 2007, Neuropsychologia.

[63]  Rajeev D. S. Raizada,et al.  Selective Amplification of Stimulus Differences during Categorical Processing of Speech , 2007, Neuron.

[64]  Bruce D. McCandliss,et al.  Brain mechanisms implicated in the preattentive categorization of speech sounds revealed using FMRI and a short-interval habituation trial paradigm. , 2007, Cerebral cortex.

[65]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[66]  A. Samuel,et al.  Perceptual adjustments to multiple speakers , 2007 .

[67]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[68]  Anne Cutler,et al.  The Dynamic Nature of Speech Perception , 2006, Language and speech.

[69]  J. McQueen,et al.  Perceptual learning in speech: stability over time. , 2006, The Journal of the Acoustical Society of America.

[70]  Emily B. Myers,et al.  The Perception of Voice Onset Time: An fMRI Investigation of Phonetic Category Structure , 2005, Journal of Cognitive Neuroscience.

[71]  A. Samuel,et al.  Perceptual learning for speech: Is there a return to normal? , 2005, Cognitive Psychology.

[72]  P. Luce,et al.  Examining the time course of indexical specificity effects in spoken word recognition. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[73]  Joanne L. Miller,et al.  Listener sensitivity to individual talker differences in voice-onset-time. , 2004, The Journal of the Acoustical Society of America.

[74]  Anne-Lise Giraud,et al.  Distinct functional substrates along the right superior temporal sulcus for the processing of voices , 2004, NeuroImage.

[75]  D. Poeppel,et al.  Dorsal and ventral streams: a framework for understanding aspects of the functional anatomy of language , 2004, Cognition.

[76]  P Sterzer,et al.  Contributions of sensory input, auditory search and verbal comprehension to cortical activity during speech processing. , 2004, Cerebral cortex.

[77]  Coarticulation • Suprasegmentals,et al.  Acoustic Phonetics , 2019, The SAGE Encyclopedia of Human Communication Sciences and Disorders.

[78]  A. Stevens,et al.  Dissociating the cortical basis of memory for voices, words and tones. , 2004, Brain research. Cognitive brain research.

[79]  R. Zatorre,et al.  Adaptation to speaker's voice in right anterior temporal lobe , 2003, Neuroreport.

[80]  A. Kleinschmidt,et al.  Modulation of neural responses to speech by directing attention to voices or verbal content. , 2003, Brain research. Cognitive brain research.

[81]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[82]  R. Newman,et al.  The perceptual consequences of within-talker variability in fricative production. , 2001, The Journal of the Acoustical Society of America.

[83]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[84]  D. Poeppel,et al.  Towards a functional neuroanatomy of speech perception , 2000, Trends in Cognitive Sciences.

[85]  R. Zatorre,et al.  Voice-selective areas in human auditory cortex , 2000, Nature.

[86]  Elizabeth A. Strand,et al.  Auditory–visual integration of talker gender in vowel perception , 1999 .

[87]  W. Fitch,et al.  Morphology and development of the human vocal tract: a study using magnetic resonance imaging. , 1999, The Journal of the Acoustical Society of America.

[88]  D. H. Slavit Phonosurgery in the Elderly: A Review , 1999, Ear, nose, & throat journal.

[89]  R. Weisskoff,et al.  Improved auditory cortex imaging using clustered volume acquisitions , 1999, Human brain mapping.

[90]  Alan C. Evans,et al.  Event-related fMRI of the auditory cortex. , 1998, NeuroImage.

[91]  S. Goldinger Words and voices: episodic traces in spoken word identification and recognition memory. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[92]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[93]  Alan C. Evans,et al.  PET studies of phonetic processing of speech: review, replication, and reanalysis. , 1996, Cerebral cortex.

[94]  J. Hillenbrand,et al.  Acoustic characteristics of American English vowels. , 1994, The Journal of the Acoustical Society of America.

[95]  D. Pisoni,et al.  Speech Perception as a Talker-Contingent Process , 1993, Psychological science.

[96]  S. Goldinger,et al.  Episodic encoding of voice attributes and recognition memory for spoken words. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[97]  D B Pisoni,et al.  Stimulus variability and processing dependencies in speech perception , 1990, Perception & psychophysics.

[98]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[99]  D. Lancker,et al.  Voice discrimination and recognition are separate abilities , 1987, Neuropsychologia.

[100]  W. Ganong Phonetic categorization in auditory word perception. , 1980, Journal of experimental psychology. Human perception and performance.

[101]  S. Boll,et al.  Suppression of acoustic noise in speech using spectral subtraction , 1979 .

[102]  A M Liberman,et al.  Perception of the speech code. , 1967, Psychological review.

[103]  G. E. Peterson,et al.  Control Methods Used in a Study of the Vowels , 1951 .