Neural Representation of Articulable and Inarticulable Novel Sound Contrasts: The Role of the Dorsal Stream

The extent that articulatory information embedded in incoming speech contributes to the formation of new perceptual categories for speech sounds has been a matter of discourse for decades. It has been theorized that the acquisition of new speech sound categories requires a network of sensory and speech motor cortical areas (the “dorsal stream”) to successfully integrate auditory and articulatory information. However, it is possible that these brain regions are not sensitive specifically to articulatory information, but instead are sensitive to the abstract phonological categories being learned. We tested this hypothesis by training participants over the course of several days on an articulable non-native speech contrast and acoustically matched inarticulable nonspeech analogues. After reaching comparable levels of proficiency with the two sets of stimuli, activation was measured in fMRI as participants passively listened to both sound types. Decoding of category membership for the articulable speech contrast alone revealed a series of left and right hemisphere regions outside of the dorsal stream that have previously been implicated in the emergence of non-native speech sound categories, while no regions could successfully decode the inarticulable nonspeech contrast. Although activation patterns in the left inferior frontal gyrus, the middle temporal gyrus, and the supplementary motor area provided better information for decoding articulable (speech) sounds compared to the inarticulable (sine wave) sounds, the finding that dorsal stream regions do not emerge as good decoders of the articulable contrast alone suggests that other factors, including the strength and structure of the emerging speech categories are more likely drivers of dorsal stream activation for novel sound learning.

[1]  Keith Johnson,et al.  Automaticity in Speech Perception: Some Speech/Nonspeech Comparisons , 1994, Phonetica.

[2]  Friedemann Pulvermüller,et al.  Motor cortex maps articulatory features of speech sounds , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Emily B. Myers Dissociable effects of phonetic competition and category typicality in a phonetic categorization task: An fMRI investigation , 2007, Neuropsychologia.

[4]  Emily B. Myers,et al.  Neural Systems Underlying Perceptual Adjustment to Non-Standard Speech Tokens. , 2014, Journal of memory and language.

[5]  Elizabeth Jefferies,et al.  The Selective Role of Premotor Cortex in Speech Perception: A Contribution to Phoneme Judgements but not Speech Comprehension , 2013, Journal of Cognitive Neuroscience.

[6]  M. Turvey,et al.  The motor theory of speech perception reviewed , 2006, Psychonomic bulletin & review.

[7]  Emily B. Myers,et al.  Effects of Category Learning on Neural Sensitivity to Non-native Phonetic Categories , 2012, Journal of Cognitive Neuroscience.

[8]  Emily B. Myers,et al.  An event-related fMRI investigation of voice-onset time discrimination , 2008, NeuroImage.

[9]  T. Carrell,et al.  Discrimination of speech-like contrasts in the auditory thalamus and cortex. , 1994, The Journal of the Acoustical Society of America.

[10]  E. Formisano,et al.  Learning of New Sound Categories Shapes Neural Response Patterns in Human Auditory Cortex , 2012, The Journal of Neuroscience.

[11]  M. Schouten,et al.  Categorical Perception as a Function of Stimulus Quality , 1999, Phonetica.

[12]  Eric Halgren,et al.  Linear Coding of Voice Onset Time , 2007, Journal of Cognitive Neuroscience.

[13]  Erika S. Levy Language experience and consonantal context effects on perceptual assimilation of French vowels by American-English learners of French. , 2009, The Journal of the Acoustical Society of America.

[14]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[15]  A J van Hessen,et al.  Categorical perception as a function of stimulus quality. , 1999, Phonetica.

[16]  Richard Granger,et al.  Categorical Speech Processing in Broca's Area: An fMRI Study Using Multivariate Pattern-Based Analysis , 2012, The Journal of Neuroscience.

[17]  Joseph C. Toscano,et al.  Continuous Perception and Graded Categorization , 2010, Psychological science.

[18]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[19]  R. Diehl,et al.  Speech Perception , 2004, Annual review of psychology.

[20]  K. Watkins,et al.  Motor Representations of Articulators Contribute to Categorical Perception of Speech Sounds , 2009, The Journal of Neuroscience.

[21]  Shinobu Masaki,et al.  Learning-induced neural plasticity associated with improved identification performance after training of a difficult second-language phonetic contrast , 2003, NeuroImage.

[22]  Marco Iacoboni,et al.  The Essential Role of Premotor Cortex in Speech Perception , 2007, Current Biology.

[23]  F. Molteni,et al.  Disentangling phonological and articulatory processing: A neuroanatomical study in aphasia , 2018, Neuropsychologia.

[24]  Martin N. Hebart,et al.  The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data , 2015, Front. Neuroinform..

[25]  Marco Iacoboni,et al.  Neural responses to non-native phonemes varying in producibility: Evidence for the sensorimotor nature of speech perception , 2006, NeuroImage.

[26]  C. Fowler An event approach to the study of speech perception from a direct realist perspective , 1986 .

[27]  Friedemann Pulvermüller,et al.  Is the Sensorimotor Cortex Relevant for Speech Perception and Understanding? An Integrative Review , 2016, Front. Hum. Neurosci..

[28]  D. Pisoni,et al.  Speech perception without traditional speech cues. , 1981, Science.

[29]  Russell A. Poldrack,et al.  The impact of study design on pattern estimation for single-trial multivariate pattern analysis , 2014, NeuroImage.

[30]  David A. Medler,et al.  Cerebral Cortex doi:10.1093/cercor/bhi040 Cerebral Cortex Advance Access published February 9, 2005 , 2022 .

[31]  A M Liberman,et al.  Perception of the speech code. , 1967, Psychological review.

[32]  Rutvik H. Desai,et al.  Specialization along the Left Superior Temporal Sulcus for Auditory Categorization , 2010, Cerebral cortex.

[33]  I. Derakhshan Cortical organization. , 2005, Journal of neurosurgery.

[34]  Adam Gazzaley,et al.  Measuring functional connectivity during distinct stages of a cognitive task , 2004, NeuroImage.

[35]  Russell A. Poldrack,et al.  Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses , 2012, NeuroImage.

[36]  B. C. Griffith,et al.  The discrimination of speech sounds within and across phoneme boundaries. , 1957, Journal of experimental psychology.

[37]  Emily B. Myers,et al.  Inferior Frontal Regions Underlie the Perception of Phonetic Category Invariance , 2009, Psychological science.

[38]  N. Kraus,et al.  An Integrative Model of Subcortical Auditory Plasticity , 2013, Brain Topography.

[39]  M. MacDonald,et al.  Production Practice During Language Learning Improves Comprehension , 2018, Psychological science.

[40]  Emily B. Myers,et al.  Sleep and native language interference affect non-native speech sound learning. , 2015, Journal of experimental psychology. Human perception and performance.

[41]  S. Jonas The thalamus and aphasia, including transcortical aphasia: a review. , 1982, Journal of communication disorders.

[42]  Jan Theeuwes,et al.  OpenSesame: An open-source, graphical experiment builder for the social sciences , 2011, Behavior Research Methods.

[43]  Arjan van Hessen,et al.  The end of categorical perception as we know it , 2003, Speech Commun..

[44]  C. Best A direct realist view of cross-language speech perception , 1995 .

[45]  Patryk A. Laurent,et al.  Evidence for Cerebellar Contributions to Adaptive Plasticity in Speech Perception. , 2015, Cerebral cortex.

[46]  Jessica S. Arsenault,et al.  No evidence of somatotopic place of articulation feature mapping in motor cortex during passive speech perception , 2016, Psychonomic bulletin & review.

[47]  D. Pisoni,et al.  Reaction times to comparisons within and across phonetic categories , 1974, Perception & psychophysics.

[48]  H. Scheich,et al.  Phonetic Perception and the Temporal Cortex , 2002, NeuroImage.

[49]  J. Hirsch,et al.  fMRI Evidence for Cortical Modification during Learning of Mandarin Lexical Tone , 2003, Journal of Cognitive Neuroscience.

[50]  Manzar Ashtari,et al.  Left middle temporal gyrus activation during a phonemic discrimination task , 2004, Neuroreport.

[51]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[52]  Jeremy D. Schmahmann,et al.  Functional topography in the human cerebellum: A meta-analysis of neuroimaging studies , 2009, NeuroImage.

[53]  P. Kuhl,et al.  Speech perception in infancy predicts language development in the second year of life: a longitudinal study. , 2004, Child development.

[54]  Matthew H. Davis,et al.  Inferior Frontal Cortex Contributions to the Recognition of Spoken Words and Their Constituent Speech Sounds , 2017, Journal of Cognitive Neuroscience.

[55]  Riikka Möttönen,et al.  Dissociating Contributions of the Motor Cortex to Speech Perception and Response Bias by Using Transcranial Magnetic Stimulation , 2014, Cerebral cortex.

[56]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[57]  D. Massaro,et al.  Categorical results do not imply categorical perception , 1982, Perception & psychophysics.

[58]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[59]  U. Ziemann,et al.  Left dorsal speech stream components and their contribution to phonological processing , 2015, Brain Stimulation.

[60]  Reflections in a mirror , 2014 .

[61]  Xin Xie,et al.  Left Inferior Frontal Gyrus Sensitivity to Phonetic Competition in Receptive Language Processing: A Comparison of Clear and Conversational Speech , 2018, Journal of Cognitive Neuroscience.

[62]  Erika S. Levy,et al.  Perception of French vowels by American English adults with and without French language experience , 2008, J. Phonetics.

[63]  Willy Serniclaes,et al.  Neural correlates of switching from auditory to speech perception , 2005, NeuroImage.

[64]  Tyler K. Perrachione,et al.  Neural characteristics of successful and less successful speech and word learning in adults , 2007, Human brain mapping.

[65]  Keith Johnson,et al.  Phonetic Feature Encoding in Human Superior Temporal Gyrus , 2014, Science.

[66]  Vincent L. Gracco,et al.  A mediating role of the premotor cortex in phoneme segmentation , 2009, Brain and Language.

[67]  Sophie K. Scott,et al.  A little more conversation, a little less action — candidate roles for the motor cortex in speech perception , 2009, Nature Reviews Neuroscience.

[68]  Stephen M. Wilson,et al.  Speech perception when the motor system is compromised , 2009, Trends in Cognitive Sciences.

[69]  Bradley R. Buchsbaum,et al.  Temporal lobe speech perception systems are part of the verbal working memory circuit: Evidence from two recent fMRI studies , 2003 .

[70]  Jessica S. Arsenault,et al.  Distributed Neural Representations of Phonological Features during Speech Perception , 2015, The Journal of Neuroscience.

[71]  D. Bates,et al.  Balancing Type I Error and Power in Linear Mixed Models , 2015, 1511.01864.

[72]  Riikka Möttönen,et al.  Discrimination of speech and non-speech sounds following theta-burst stimulation of the motor cortex , 2014, Front. Psychol..

[73]  Lori L. Holt,et al.  Reflections on mirror neurons and speech perception , 2009, Trends in Cognitive Sciences.

[74]  Chris I. Baker,et al.  Deconstructing multivariate decoding for the study of brain function , 2017, NeuroImage.

[75]  Joan López-Moliner,et al.  quickpsy: An R Package to Fit Psychometric Functions for Multiple Groups , 2016, R J..

[76]  Jeffrey R. Binder,et al.  Left Posterior Temporal Regions are Sensitive to Auditory Categorization , 2008, Journal of Cognitive Neuroscience.

[77]  M. Iacoboni,et al.  Listening to speech activates motor areas involved in speech production , 2004, Nature Neuroscience.

[78]  Laurent Dekydtspotter,et al.  Direct mapping of acoustics to phonology: On the lexical encoding of front rounded vowels in L1 English– L2 French acquisition , 2012 .

[79]  M. Schouten,et al.  Categorical perception depends on the discrimination task , 2004, Perception & psychophysics.

[80]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[81]  D B Pisoni,et al.  Some experiments on perceptual learning of mirror-image acoustic patterns , 1982, Perception & psychophysics.

[82]  Milene Bonte,et al.  Decoding Articulatory Features from fMRI Responses in Dorsal Speech Regions , 2015, The Journal of Neuroscience.

[83]  Claus-W. Wallesch,et al.  Lesions of the basal ganglia, thalamus, and deep white matter: Differential effects on language functions , 1983, Brain and Language.

[84]  Susan M. Garnsey,et al.  The time-course of cortical responses to speech revealed by fast optical imaging , 2018, Brain and Language.

[85]  Emily B. Myers,et al.  The Perception of Voice Onset Time: An fMRI Investigation of Phonetic Category Structure , 2005, Journal of Cognitive Neuroscience.