Control of Spoken Vowel Acoustics and the Influence of Phonetic Context in Human Speech Sensorimotor Cortex

Speech production requires the precise control of vocal tract movements to generate individual speech sounds (phonemes) which, in turn, are rapidly organized into complex sequences. Multiple productions of the same phoneme can exhibit substantial variability, some of which is inherent to control of the vocal tract and its biomechanics, and some of which reflects the contextual effects of surrounding phonemes (“coarticulation”). The role of the CNS in these aspects of speech motor control is not well understood. To address these issues, we recorded multielectrode cortical activity directly from human ventral sensory-motor cortex (vSMC) during the production of consonant-vowel syllables. We analyzed the relationship between the acoustic parameters of vowels (pitch and formants) and cortical activity on a single-trial level. We found that vSMC activity robustly predicted acoustic parameters across vowel categories (up to 80% of variance), as well as different renditions of the same vowel (up to 25% of variance). Furthermore, we observed significant contextual effects on vSMC representations of produced phonemes that suggest active control of coarticulation: vSMC representations for vowels were biased toward the representations of the preceding consonant, and conversely, representations for consonants were biased toward upcoming vowels. These results reveal that vSMC activity for phonemes are not invariant and provide insight into the cortical mechanisms of coarticulation.

[1]  M. Sahani,et al.  Cortical control of arm movements: a dynamical systems perspective. , 2013, Annual review of neuroscience.

[2]  K. Shenoy,et al.  A Central Source of Movement Variability , 2006, Neuron.

[3]  Louis Goldstein,et al.  Articulatory gestures as phonological units , 1989, Phonology.

[4]  Yuichi Ueda,et al.  A real-time formant tracker based on the inverse filter control method , 2007 .

[5]  Steven Brown,et al.  Representation of the speech effectors in the human motor cortex: Somatotopy or overlap? , 2010, Brain and Language.

[6]  Michael S. Brainard,et al.  Central Contributions to Acoustic Variation in Birdsong , 2008, The Journal of Neuroscience.

[7]  J. Maunsell,et al.  Different Origins of Gamma Rhythm and High-Gamma Activity in Macaque Visual Cortex , 2011, PLoS biology.

[8]  Patricia A. Keating,et al.  Phonetic representations in a generative grammar , 1990 .

[9]  Peter F. MacNeilage,et al.  The Production of Speech , 2011, Springer New York.

[10]  Nicholas P. Szrama,et al.  Using the electrocorticographic speech network to control a brain–computer interface in humans , 2011, Journal of neural engineering.

[11]  Joseph S. Perkell,et al.  Coarticulation strategies: preliminary implications of a detailed analysis of lower lip protrusion movements , 1986, Speech Commun..

[12]  Willem J. M. Levelt,et al.  Producing spoken language: A blueprint of the speaker , 1999 .

[13]  J. Hillenbrand,et al.  Acoustic characteristics of American English vowels. , 1994, The Journal of the Acoustical Society of America.

[14]  Daniel Bullock,et al.  Neural Representations and Mechanisms for the Performance of Simple Speech Sequences , 2010, Journal of Cognitive Neuroscience.

[15]  L. A. Chistovich,et al.  Speech: articulation and perception , 1965 .

[16]  Robert T. Knight,et al.  Spatiotemporal imaging of cortical activation during verb generation and picture naming , 2010, NeuroImage.

[17]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[18]  B. Lindblom Spectrographic Study of Vowel Reduction , 1963 .

[19]  Daniel Bullock,et al.  Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives. , 2004, Human movement science.

[20]  John P. Cunningham,et al.  Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity , 2008, NIPS.

[21]  M. Halle,et al.  Preliminaries to Speech Analysis: The Distinctive Features and Their Correlates , 1961 .

[22]  D. Whalen Coarticulation is largely planned , 1990 .

[23]  Akira Watanabe,et al.  Formant estimation method using inverse-filter control , 2001, IEEE Trans. Speech Audio Process..

[24]  Byron M. Yu,et al.  Techniques for extracting single-trial activity patterns from large-scale neural recordings , 2007, Current Opinion in Neurobiology.

[25]  R. Daniloff,et al.  Coarticulation of lip rounding. , 1968, Journal of speech and hearing research.

[26]  D J Ostry,et al.  Coarticulation of jaw movements in speech production: is context sensitivity in speech kinematics centrally planned? , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  G. Laurent,et al.  Transient Dynamics versus Fixed Points in Odor Representations by Locust Antennal Lobe Projection Neurons , 2005, Neuron.

[28]  Frank H. Guenther,et al.  An fMRI investigation of syllable sequence production , 2006, NeuroImage.

[29]  Kristofer E. Bouchard,et al.  Functional Organization of Human Sensorimotor Cortex for Speech Articulation , 2013, Nature.

[30]  Katherine I. Nagel,et al.  Cortical Mechanisms of Smooth Eye Movements Revealed by Dynamic Covariations of Neural and Behavioral Responses , 2008, Neuron.

[31]  Kevin L. Briggman,et al.  Imaging Dedicated and Multifunctional Neural Circuits Generating Distinct Behaviors , 2006, The Journal of Neuroscience.

[32]  Raymond D. Kent,et al.  Coarticulation in recent speech production models , 1977 .

[33]  Eric Leuthardt,et al.  Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition , 2011, NeuroImage.

[34]  P. Ladefoged A course in phonetics , 1975 .

[35]  Peter Q. Pfordresher,et al.  The somatotopy of speech: Phonation and articulation in the human motor cortex , 2009, Brain and Cognition.

[36]  J. Abbs,et al.  Variant and invariant characteristics of speech movements , 2004, Experimental Brain Research.

[37]  S. Ohman Coarticulation in VCV utterances: spectrographic measurements. , 1966, The Journal of the Acoustical Society of America.

[38]  Dezhe Z. Jin,et al.  Support for a synaptic chain model of neuronal sequence generation , 2010, Nature.

[39]  J. Perkell,et al.  Variability in production of the vowels /i/ and /a/. , 1985, The Journal of the Acoustical Society of America.

[40]  Jason A. Tourville,et al.  The integration of large-scale neural network modeling and functional brain imaging in speech motor control , 2010, NeuroImage.

[41]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[42]  F. Guenther,et al.  A Wireless Brain-Machine Interface for Real-Time Speech Synthesis , 2009, PloS one.

[43]  Kevin L. Briggman,et al.  Optical Imaging of Neuronal Populations During Decision-Making , 2005, Science.

[44]  Björn Lindblom,et al.  Economy of Speech Gestures , 1983 .

[45]  Michael S Brainard,et al.  Linked Control of Syllable Sequence and Phonology in Birdsong , 2010, The Journal of Neuroscience.

[46]  Byron M. Yu,et al.  Single-Trial Neural Correlates of Arm Movement Preparation , 2011, Neuron.

[47]  R. Lesser,et al.  Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. , 1998, Brain : a journal of neurology.

[48]  Byron M. Yu,et al.  Neural Variability in Premotor Cortex Provides a Signature of Motor Preparation , 2006, The Journal of Neuroscience.

[49]  Satrajit S. Ghosh,et al.  Neural modeling and imaging of the cortical interactions underlying syllable production , 2006, Brain and Language.

[50]  R. Lesser,et al.  Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. I. Alpha and beta event-related desynchronization. , 1998, Brain : a journal of neurology.

[51]  F H Guenther,et al.  Speech sound acquisition, coarticulation, and rate effects in a neural network model of speech production. , 1995, Psychological review.

[52]  Christian Abry,et al.  Test of the movement expansion model: anticipatory vowel lip protrusion and constriction in French and English speakers. , 2011, The Journal of the Acoustical Society of America.

[53]  Noam Chomsky,et al.  The Sound Pattern of English , 1968 .

[54]  Kelvin E. Jones,et al.  Sources of signal-dependent noise during isometric force production. , 2002, Journal of neurophysiology.

[55]  Bruno B Averbeck,et al.  Parallel processing of serial movements in prefrontal cortex , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[56]  Carol A. Fowler,et al.  Coarticulation and theories of extrinsic timing , 1980 .