Vocal Tract Images Reveal Neural Representations of Sensorimotor Transformation During Speech Imitation

&NA; Imitating speech necessitates the transformation from sensory targets to vocal tract motor output, yet little is known about the representational basis of this process in the human brain. Here, we address this question by using real‐time MR imaging (rtMRI) of the vocal tract and functional MRI (fMRI) of the brain in a speech imitation paradigm. Participants trained on imitating a native vowel and a similar nonnative vowel that required lip rounding. Later, participants imitated these vowels and an untrained vowel pair during separate fMRI and rtMRI runs. Univariate fMRI analyses revealed that regions including left inferior frontal gyrus were more active during sensorimotor transformation (ST) and production of nonnative vowels, compared with native vowels; further, ST for nonnative vowels activated somatomotor cortex bilaterally, compared with ST of native vowels. Using test representational similarity analysis (RSA) models constructed from participants’ vocal tract images and from stimulus formant distances, we found that RSA searchlight analyses of fMRI data showed either type of model could be represented in somatomotor, temporal, cerebellar, and hippocampal neural activation patterns during ST. We thus provide the first evidence of widespread and robust cortical and subcortical neural representation of vocal tract and/or formant parameters, during prearticulatory ST.

[1]  Rachel A Diana,et al.  High‐resolution multi‐voxel pattern analysis of category selectivity in the medial temporal lobes , 2008, Hippocampus.

[2]  Marco Iacoboni,et al.  Neural responses to non-native phonemes varying in producibility: Evidence for the sensorimotor nature of speech perception , 2006, NeuroImage.

[3]  Kate E Watkins,et al.  Changes in neural activity associated with learning to articulate novel auditory pseudowords by covert repetition , 2008, Human brain mapping.

[4]  Marzena Wylezinska,et al.  Speech MRI: morphology and function. , 2014, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[5]  Yinjuan Du,et al.  Noise differentially impacts phoneme representations in the auditory and speech motor systems , 2014, Proceedings of the National Academy of Sciences.

[6]  S. Kastner,et al.  Complex organization of human primary motor cortex: a high-resolution fMRI study. , 2008, Journal of neurophysiology.

[7]  G. Hickok Computational neuroanatomy of speech production , 2012, Nature Reviews Neuroscience.

[8]  K. Watkins,et al.  Motor Representations of Articulators Contribute to Categorical Perception of Speech Sounds , 2009, The Journal of Neuroscience.

[9]  A M Liberman,et al.  Perception of the speech code. , 1967, Psychological review.

[10]  Cathy J. Price,et al.  Sensory-to-motor integration during auditory repetition: a combined fMRI and lesion study , 2014, Front. Hum. Neurosci..

[11]  Kristofer E. Bouchard,et al.  Functional Organization of Human Sensorimotor Cortex for Speech Articulation , 2013, Nature.

[12]  M. Erb,et al.  The influence of syllable onset complexity and syllable frequency on speech motor control , 2008, Brain and Language.

[13]  Kayoko Okada,et al.  Area Spt in the Human Planum Temporale Supports Sensory-motor Integration for Speech Processing Establishing the Existence of Distinct Sen- Sory versus Motor Activation Patterns Would Establish That , 2022 .

[14]  S. Nagarajan,et al.  What Does Motor Efference Copy Represent? Evidence from Speech Production , 2013, The Journal of Neuroscience.

[15]  Leslie G. Ungerleider,et al.  Experience-dependent changes in cerebellar contributions to motor sequence learning , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[16]  Edward F Chang,et al.  The auditory representation of speech sounds in human motor cortex , 2016, eLife.

[17]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Friedemann Pulvermüller,et al.  Motor cortex maps articulatory features of speech sounds , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[19]  L. Fadiga,et al.  The Motor Somatotopy of Speech Perception , 2009, Current Biology.

[20]  Jessica S. Arsenault,et al.  Distributed Neural Representations of Phonological Features during Speech Perception , 2015, The Journal of Neuroscience.

[21]  M. Hautus Corrections for extreme proportions and their biasing effects on estimated values ofd′ , 1995 .

[22]  Denis G. Pelli,et al.  ECVP '07 Abstracts , 2007, Perception.

[23]  Alexis Hervais-Adelman,et al.  The effect of phonetic production training with visual feedback on the perception and production of foreign speech sounds. , 2015, The Journal of the Acoustical Society of America.

[24]  Shrikanth S. Narayanan,et al.  Enhanced airway-tissue boundary segmentation for real-time magnetic resonance imaging data , 2014 .

[25]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[26]  Matthew H. Davis,et al.  Learning and Consolidation of Novel Spoken Words , 2009, Journal of Cognitive Neuroscience.

[27]  Robert Leech,et al.  A comparison of sensory-motor activity during speech in first and second languages. , 2011, Journal of neurophysiology.

[28]  J. Rothwell,et al.  Speech Facilitation by Left Inferior Frontal Cortex Stimulation , 2011, Current Biology.

[29]  Eraldo Paulesu,et al.  The role of age of acquisition and language usage in early, high‐proficient bilinguals: An fMRI study during verbal fluency , 2003, Human brain mapping.

[30]  Matthew H. Davis,et al.  Hierarchical Organization of Auditory and Motor Representations in Speech Perception: Evidence from Searchlight Similarity Analysis , 2015, Cerebral cortex.

[31]  Frank H. Guenther,et al.  A neural theory of speech acquisition and production , 2012, Journal of Neurolinguistics.

[32]  Robert Leech,et al.  Overlapping Networks Engaged during Spoken Language Production and Its Cognitive Control , 2014, The Journal of Neuroscience.

[33]  Laurent Lamalle,et al.  Functional MRI assessment of orofacial articulators: Neural correlates of lip, jaw, larynx, and tongue movements , 2012, Human brain mapping.

[34]  Christopher J. Markiewicz,et al.  Mapping the cortical representation of speech sounds in a syllable repetition task , 2016, NeuroImage.

[35]  F. Guenther Cortical interactions underlying the production of speech sounds. , 2006, Journal of communication disorders.

[36]  Chris Rorden,et al.  Neural recruitment for the production of native and novel speech sounds , 2009, NeuroImage.

[37]  Keith Johnson,et al.  Phonetic Feature Encoding in Human Superior Temporal Gyrus , 2014, Science.

[38]  Steven Brown,et al.  Representation of the speech effectors in the human motor cortex: Somatotopy or overlap? , 2010, Brain and Language.

[39]  Andrew D Scott,et al.  Adaptive averaging applied to dynamic imaging of the soft palate , 2013, Magnetic resonance in medicine.

[40]  Eleanor A. Maguire,et al.  Representations of specific acoustic patterns in the auditory cortex and hippocampus , 2014, Proceedings of the Royal Society B: Biological Sciences.

[41]  Bijan Pesaran,et al.  Sensory-motor transformations for speech occur bilaterally , 2014, Nature.

[42]  Matthew H. Davis,et al.  Hierarchical Processing in Spoken Language Comprehension , 2003, The Journal of Neuroscience.

[43]  Marc F. Joanisse,et al.  Sensitivity of human auditory cortex to rapid frequency modulation revealed by multivariate representational similarity analysis , 2014, Front. Neurosci..

[44]  Sophie K. Scott,et al.  Neural Correlates of Sublexical Processing in Phonological Working Memory , 2011, Journal of Cognitive Neuroscience.

[45]  Robert J Zatorre,et al.  Learning new sounds of speech: reallocation of neural substrates , 2004, NeuroImage.

[46]  Bradley R. Buchsbaum,et al.  Temporal lobe speech perception systems are part of the verbal working memory circuit: Evidence from two recent fMRI studies , 2003 .

[47]  Bradley P Sutton,et al.  Simultaneous dynamic and functional MRI scanning (SimulScan) of natural swallows , 2011, Magnetic resonance in medicine.

[48]  Daniel Bullock,et al.  Neural Representations and Mechanisms for the Performance of Simple Speech Sequences , 2010, Journal of Cognitive Neuroscience.

[49]  D. Carey,et al.  Magnetic resonance imaging of the brain and vocal tract: Applications to the study of speech production and language learning , 2017, Neuropsychologia.

[50]  S. Scott,et al.  The Pathways for Intelligible Speech: Multivariate and Univariate Perspectives , 2013, Cerebral cortex.

[51]  A. Simmonds,et al.  A hypothesis on improving foreign accents by optimizing variability in vocal learning brain circuits , 2015, Front. Hum. Neurosci..

[52]  N. Kriegeskorte,et al.  Author ' s personal copy Representational geometry : integrating cognition , computation , and the brain , 2013 .

[53]  Kristofer E. Bouchard,et al.  High-Resolution, Non-Invasive Imaging of Upper Vocal Tract Articulators Compatible with Human Brain Recordings , 2016, PloS one.

[54]  Jens Frahm,et al.  Real‐time MRI of speaking at a resolution of 33 ms: Undersampled radial FLASH with nonlinear inverse reconstruction , 2013, Magnetic resonance in medicine.

[55]  Paul Iverson,et al.  The response of the anterior striatum during adult human vocal learning. , 2014, Journal of neurophysiology.

[56]  Jason A. Tourville,et al.  The Neural Correlates of Speech Motor Sequence Learning , 2015, Journal of Cognitive Neuroscience.

[57]  M. D’Esposito,et al.  Medial Temporal Lobe Activity Associated with Active Maintenance of Novel Information , 2001, Neuron.

[58]  F. Dick,et al.  Functional and Quantitative MRI Mapping of Somatomotor Representations of Human Supralaryngeal Vocal Tract , 2017, Cerebral cortex.

[59]  William G. Pearson,et al.  Effortful pitch glide: a potential new exercise evaluated by dynamic MRI. , 2014, Journal of speech, language, and hearing research : JSLHR.

[60]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[61]  Milene Bonte,et al.  Decoding Articulatory Features from fMRI Responses in Dorsal Speech Regions , 2015, The Journal of Neuroscience.

[62]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[63]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[64]  J. Wells Local accents in England and Wales , 1970, Journal of Linguistics.

[65]  Matthew K. Leonard,et al.  The peri-Sylvian cortical network underlying single word repetition revealed by electrocortical stimulation and direct neural recordings , 2016, Brain and Language.

[66]  Marc E Miquel,et al.  Recommendations for real‐time speech MRI , 2016, Journal of magnetic resonance imaging : JMRI.

[67]  E. Ngan,et al.  A larynx area in the human motor cortex. , 2008, Cerebral cortex.

[68]  Douglas D. O'Shaughnessy,et al.  Speech communication : human and machine , 1987 .

[69]  Robert Leech,et al.  Sensory-Motor Integration during Speech Production Localizes to Both Left and Right Plana Temporale , 2014, The Journal of Neuroscience.

[70]  Shrikanth S. Narayanan,et al.  Advances in real-time magnetic resonance imaging of the vocal tract for speech science and technology research , 2016, APSIPA Transactions on Signal and Information Processing.