Integrating Linguistic, Motor, and Perceptual Information in Language Production.

Speakers show remarkable adaptability in updating and correcting their utterances in response to changes in the environment. When an interlocutor raises an eyebrow or the AC kicks on and introduces ambient noise, it seems that speakers are able to quickly integrate this information into their speech plans and adapt appropriately. This ability to update a plan on the fly has been observed in other motor behaviors as well, classically demonstrated in studies of adaptation to prism glasses. Environmental factors aren’t the only information sources speakers monitor, though. Speakers also have access to propioceptive feedback from their own articulatory gestures, perceptual feedback as they hear their own voices, and the conceptual and linguistic content of the intended message. How do these information sources get combined during language production to result in an utterance that successfully balances constraints imposed by the environment and cognitive processing with the speaker’s communicative goals? I address these issues through several analyses of a study of sensorimotor learning in response to perceptual perturbation. Speakers read single words aloud as prompted by a computer display. During their speech, realtime signal processing is used to measure and adjust the value of the first formant (F1) in the auditory feedback available to the speaker. Due to this perturbation, each speaker heard the sound of the vowel /E/ (as in "bed") as either /I/ ("bid") or /æ/ ("bad"). Previous studies have shown that when exposed to this kind of perturbation, speakers quickly adapt by producing a vowel that has an F1 that is shifted in the opposite direction. The words produced in the present study differed as to whether the target word had a "spoken competitor" or a "heard competitor". If the F1 /E/ is shifted down towards /I/, "bed" will have a "heard competitor" of "bid". To counteract this perturbation, speakers are predicted to adapt by producing a more /æ/-like vowel, resulting in something like "bad". In

[1]  W. Levelt,et al.  Monitoring the Time Course of Phonological Encoding , 1995 .

[2]  J. K. Bock.  Syntactic persistence in language production , 1986, Cognitive Psychology.

[3]  Scott T. Grafton,et al.  Forward modeling allows feedback control for fast reaching movements , 2000, Trends in Cognitive Sciences.

[4]  H. Lane,et al.  The Lombard Sign and the Role of Hearing in Speech , 1971 .

[5]  P. Luce,et al.  Probabilistic Phonotactics and Neighborhood Activation in Spoken Word Recognition , 1999 .

[6]  D. Pisoni,et al.  Recognizing Spoken Words: The Neighborhood Activation Model , 1998, Ear and hearing.

[7]  M. Garrett.  Levels of processing in sentence production , 1980 .

[8]  James S. Magnuson,et al.  The Dynamics of Lexical Competition During Spoken Word Recognition , 2007, Cogn. Sci..

[9]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[10]  G. Dell,et al.  Language production and serial order: a functional analysis and a model. , 1997, Psychological review.

[11]  M. Tanenhaus,et al.  The time course of spoken word learning and recognition: studies with artificial lexicons. , 2003, Journal of experimental psychology. General.

[12]  K. Munhall,et al.  Compensation following real-time manipulation of formants in isolated vowels. , 2006, The Journal of the Acoustical Society of America.

[13]  T. Jaeger,et al.  Categorical Data Analysis: Away from ANOVAs (transformation or not) and towards Logit Mixed Models. , 2008, Journal of memory and language.

[14]  R. Jacobs,et al.  Perception of speech reflects optimal use of probabilistic speech cues , 2008, Cognition.

[15]  R. Harald Baayen,et al.  Statistics in Psycholinguistics: A critique of some current gold standards , 2004 .

[16]  H. Wallach,et al.  A PASSIVE CONDITION FOR RAPID ADAPTATION TO DISPLACED VISUAL DIRECTION. , 1963, The American journal of psychology.

[17]  V A Fromkin.  Slips of the tongue. , 1973, Scientific American.

[18]  Marianne Pouplier,et al.  Tongue Kinematics during Utterances Elicited with the SLIP Technique , 2007, Language and speech.

[19]  G. Dell,et al.  The sequential cuing effect in speech production , 1994, Cognition.

[20]  R. Baayen,et al.  Putting the bits together: an information theoretical perspective on morphological processing , 2004, Cognition.

[21]  W. Marslen-Wilson.  Functional parallelism in spoken word-recognition , 1987, Cognition.

[22]  Paul D. Allopenna,et al.  Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models , 1998 .

[23]  Joseph Paul Stemberger,et al.  Neighbourhood effects on error rates in speech production , 2004, Brain and Language.

[24]  Joseph S Perkell,et al.  Time course of speech changes in response to unanticipated short-term changes in hearing state. , 2007, The Journal of the Acoustical Society of America.

[25]  M. Vitevitch,et al.  Sublexical and lexical representations in speech production: effects of phonotactic probability and onset density. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[26]  G. E. Peterson,et al.  Control Methods Used in a Study of the Vowels , 1951 .

[27]  Gary S. Dell,et al.  Inner speech slips exhibit lexical bias, but not the phonemic similarity effect , 2008, Cognition.

[28]  David A. Belsley,et al.  Regression Analysis and its Application: A Data-Oriented Approach.@@@Applied Linear Regression.@@@Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1981 .

[29]  Zenzi M. Griffin,et al.  The persistence of structural priming: transient activation or implicit learning? , 2000, Journal of experimental psychology. General.

[30]  Richard N Aslin,et al.  Tracking the time course of phonetic cue integration during spoken word recognition , 2008, Psychonomic bulletin & review.

[31]  Albert Costa,et al.  Neighbourhood density and frequency effects in speech production: A case for interactivity , 2008 .

[32]  Louis Goldstein,et al.  Gestural specification using dynamically-defined articulatory structures , 1990 .

[33]  Martin Corley,et al.  Articulatory evidence for feedback and competition in speech production , 2009 .

[34]  Robert Tibshirani.  Bootstrap Confidence Intervals , 1984 .

[35]  Oral feedback I. Variability of the effect of nerve-block anesthesia upon speech , 1973 .

[36]  Jan P. H. van Santen,et al.  Duration and spectral balance of intervocalic consonants: A case for efficient communication , 2005, Speech Commun..

[37]  P. Luce,et al.  When Words Compete: Levels of Processing in Perception of Spoken Words , 1998 .

[38]  David J. Ostry,et al.  Auditory plasticity and speech motor learning , 2009, Proceedings of the National Academy of Sciences.

[39]  Kenneth N. Stevens,et al.  Features in Speech Perception and Lexical Access , 2008 .

[40]  S. Blumstein,et al.  Cascading activation from phonological planning to articulatory processes: Evidence from tongue twisters , 2006 .

[41]  A. Gelman.  Scaling regression inputs by dividing by two standard deviations , 2008, Statistics in medicine.

[42]  Michael I. Jordan,et al.  Sensorimotor adaptation of speech I: Compensation and adaptation. , 2002, Journal of speech, language, and hearing research : JSLHR.

[43]  Jeffery A. Jones,et al.  Remapping Auditory-Motor Representations in Voice Production , 2005, Current Biology.

[44]  Alice Turk,et al.  The Smooth Signal Redundancy Hypothesis: A Functional Explanation for Relationships between Redundancy, Prosodic Prominence, and Duration in Spontaneous Speech , 2004, Language and speech.

[45]  Victor S. Ferreira,et al.  The persistence of optional complementizer production: Why saying “that” is not saying “that” at all , 2003 .

[46]  A. Roelofs,et al.  Spoken language planning and the initiation of articulation , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[47]  F. Guenther,et al.  A theoretical investigation of reference frames for the planning of speech movements. , 1998, Psychological review.

[48]  Kevin G Munhall,et al.  Adaptive control of vowel formant frequency: evidence from real-time formant manipulation. , 2006, The Journal of the Acoustical Society of America.

[49]  Michael S Vitevitch,et al.  The influence of phonological similarity neighborhoods on speech production. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[50]  Dennis Norris,et al.  The Bayesian reader: explaining word recognition as an optimal Bayesian decision process. , 2006, Psychological review.

[51]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[52]  C. Fowler,et al.  Talkers' signaling of new and old. words in speech and listeners' perception and use of the distinction , 1987 .

[53]  G. Dell,et al.  Effect of Ambiguity and Lexical Availability on Syntactic and Lexical Production , 2000, Cognitive Psychology.

[54]  A. Roelofs,et al.  The WEAVER model of word-form encoding in speech production , 1997, Cognition.

[55]  Jan P. H. van Santen,et al.  Strong interaction between factors influencing consonant duration , 1997, EUROSPEECH.

[56]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[57]  J. Perkell,et al.  Sensorimotor adaptation to feedback perturbations of vowel acoustics and its relation to perception. , 2007, The Journal of the Acoustical Society of America.

[58]  Jeffrey L. Elman,et al.  Coherence and Coreference Revisited , 2007, J. Semant..

[59]  Mirjam Ernestus,et al.  Lexical frequency and acoustic reduction in spoken Dutch. , 2005, The Journal of the Acoustical Society of America.

[60]  G S Dell,et al.  A spreading-activation theory of retrieval in sentence production. , 1986, Psychological review.

[61]  Michael K. Tanenhaus,et al.  The Weckud Wetch of the Wast: Lexical Adaptation to a Novel Accent , 2008, Cogn. Sci..

[62]  Yi Xu,et al.  Compensation for pitch-shifted auditory feedback during the production of Mandarin tone sequences. , 2004, The Journal of the Acoustical Society of America.

[63]  Jeffery A. Jones,et al.  Perceptual calibration of F0 production: evidence from feedback perturbation. , 2000, The Journal of the Acoustical Society of America.

[64]  V. Ferreira.  Is It Better to Give Than to Donate? Syntactic Flexibility in Language Production , 1996 .

[65]  Joseph S. Perkell.  Sensory Goals and Control Mechanisms for Phonemic Articulations , 2007 .

[66]  R. Jacobs,et al.  Optimal integration of texture and motion cues to depth , 1999, Vision Research.

[67]  Willem J. M. Levelt,et al.  A theory of lexical access in speech production , 1999, Behavioral and Brain Sciences.

[68]  E. Bard,et al.  Controlling the Intelligibility of Referring Expressions in Dialogue , 2000 .

[69]  J. Abbs,et al.  Control of complex motor gestures: orofacial muscle responses to load perturbations of lip during speech. , 1984, Journal of neurophysiology.

[70]  Elliot Saltzman,et al.  Dynamics and coordinate systems in skilled sensorimotor activity , 1996 .

[71]  Michael I. Jordan,et al.  Sensorimotor adaptation in speech production. , 1998, Science.

[72]  D. Pisoni,et al.  Training Japanese listeners to identify English /r/ and /l/: a first report. , 1991, The Journal of the Acoustical Society of America.