What drives the perceptual change resulting from speech motor adaptation? Evaluation of hypotheses in a Bayesian modeling framework

Shifts in perceptual boundaries resulting from speech motor learning induced by perturbations of the auditory feedback were taken as evidence for the involvement of motor functions in auditory speech perception. Beyond this general statement, the precise mechanisms underlying this involvement are not yet fully understood. In this paper we propose a quantitative evaluation of some hypotheses concerning the motor and auditory updates that could result from motor learning, in the context of various assumptions about the roles of the auditory and somatosensory pathways in speech perception. This analysis was made possible thanks to the use of a Bayesian model that implements these hypotheses by expressing the relationships between speech production and speech perception in a joint probability distribution. The evaluation focuses on how the hypotheses can (1) predict the location of perceptual boundary shifts once the perturbation has been removed, (2) account for the magnitude of the compensation in presence of the perturbation, and (3) describe the correlation between these two behavioral characteristics. Experimental findings about changes in speech perception following adaptation to auditory feedback perturbations serve as reference. Simulations suggest that they are compatible with a framework in which motor adaptation updates both the auditory-motor internal model and the auditory characterization of the perturbed phoneme, and where perception involves both auditory and somatosensory pathways.

[1]  Daniel R. Lametti,et al.  Plasticity in the Human Speech Motor System Drives Changes in Speech Perception , 2014, The Journal of Neuroscience.

[2]  C A Fowler,et al.  Listeners do hear sounds, not tongues. , 1996, The Journal of the Acoustical Society of America.

[3]  Julien Diard Bayesian Algorithmic Modeling in Cognitive Science , 2015 .

[4]  Konrad Paul Kording,et al.  Relevance of error: what drives motor adaptation? , 2009, Journal of neurophysiology.

[5]  Scott E. Bevans,et al.  Effect of visual error size on saccade adaptation in monkey. , 2003, Journal of neurophysiology.

[6]  A. J. Yates Delayed Auditory Feedback and Shadowing , 1965 .

[7]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[8]  J. McQueen,et al.  Mapping the Speech Code: Cortical Responses Linking the Perception and Production of Vowels , 2017, Front. Hum. Neurosci..

[9]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[10]  Minoru Sasaki,et al.  Computational model of motor learning and perceptual change , 2013, Biological Cybernetics.

[11]  Pascal Perrier,et al.  Does Auditory-Motor Learning of Speech Transfer from the CV Syllable to the CVCV Word? , 2016, INTERSPEECH.

[12]  Dave F. Kleinschmidt,et al.  Robust speech perception: recognize the familiar, generalize to the similar, and adapt to the novel. , 2015, Psychological review.

[13]  Xiaoqin Wang,et al.  Neural substrates of vocalization feedback monitoring in primate auditory cortex , 2008, Nature.

[14]  Satrajit S. Ghosh,et al.  Focal Manipulations of Formant Trajectories Reveal a Role of Auditory Feedback in the Online Control of Both Within-Syllable and Between-Syllable Speech Timing , 2011, The Journal of Neuroscience.

[15]  D. Henriques,et al.  Sensory recalibration of hand position following visuomotor adaptation. , 2009, Journal of neurophysiology.

[16]  Jean-Luc Schwartz,et al.  Sensorimotor learning in a Bayesian computational model of speech communication , 2016, 2016 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[17]  M. Merzenich,et al.  Modulation of the Auditory Cortex during Speech: An MEG Study , 2002, Journal of Cognitive Neuroscience.

[18]  Julien Diard,et al.  Bayesian Modeling in Speech Motor Control: A Principled Structure for the Integration of Various Constraints , 2016, INTERSPEECH.

[19]  Michael I. Jordan,et al.  Sensorimotor adaptation of speech I: Compensation and adaptation. , 2002, Journal of speech, language, and hearing research : JSLHR.

[20]  Jason A. Tourville,et al.  Neural mechanisms underlying auditory feedback control of speech , 2008, NeuroImage.

[21]  Satrajit S. Ghosh,et al.  fMRI investigation of unexpected somatosensory feedback perturbation during speech , 2011, NeuroImage.

[22]  Joseph T. Devlin,et al.  The hearing ear is always found close to the speaking tongue: Review of the role of the motor system in speech perception , 2017, Brain and Language.

[23]  Jeremy D Wong,et al.  Somatosensory Plasticity and Motor Learning , 2010, The Journal of Neuroscience.

[24]  Kenneth N. Stevens,et al.  On the quantal nature of speech , 1972 .

[25]  Kamel Mekhnacha,et al.  Bayesian Programming , 2013 .

[26]  R. Jacobs,et al.  Perception of speech reflects optimal use of probabilistic speech cues , 2008, Cognition.

[27]  C. Fowler An event approach to the study of speech perception from a direct realist perspective , 1986 .

[28]  Kevin G. Munhall,et al.  Functional Overlap between Regions Involved in Speech Perception and in Monitoring One's Own Voice during Speech Production , 2010, Journal of Cognitive Neuroscience.

[29]  A. Friederici The cortical language circuit: from auditory perception to sentence comprehension , 2012, Trends in Cognitive Sciences.

[30]  K. Sakai,et al.  Brain activations during conscious self‐monitoring of speech production with delayed auditory feedback: An fMRI study , 2003, Human brain mapping.

[31]  V. Gracco,et al.  Perceptual recalibration of speech sounds following speech motor learning. , 2009, The Journal of the Acoustical Society of America.

[32]  Naomi H. Feldman,et al.  The influence of categories on perception: explaining the perceptual magnet effect as optimal statistical inference. , 2009, Psychological review.

[33]  Julien Diard,et al.  Modélisation bayésienne de la planification motrice des gestes de parole: Évaluation du rôle des différentes modalités sensorielles (Bayesian modeling of speech gesture motor planning: Evaluating the role of different sensory modalities )[In French] , 2016, JEPTALNRECITAL.

[34]  Julien Diard,et al.  Bayesian Action–Perception Computational Model: Interaction of Production and Recognition of Cursive Letters , 2011, PloS one.

[35]  S. Blumstein,et al.  Phonetic features and acoustic invariance in speech , 1981, Cognition.

[36]  Jean-Luc Schwartz,et al.  Adverse conditions improve distinguishability of auditory, motor, and perceptuo-motor theories of speech perception: An exploratory Bayesian modelling study , 2012 .

[37]  Michael I. Jordan,et al.  Sensorimotor adaptation in speech production. , 1998, Science.

[38]  Jean-Luc Schwartz,et al.  Assessing Idiosyncrasies in a Bayesian Model of Speech Communication , 2016, INTERSPEECH.

[39]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[40]  Satrajit S. Ghosh,et al.  Adaptive auditory feedback control of the production of formant trajectories in the Mandarin triphthong /iau/ and its pattern of generalization. , 2010, The Journal of the Acoustical Society of America.

[41]  Bart de Boer,et al.  Investigating the role of infant-directed speech with a computer model , 2003 .

[42]  T. Poggio,et al.  BOOK REVIEW David Marr’s Vision: floreat computational neuroscience VISION: A COMPUTATIONAL INVESTIGATION INTO THE HUMAN REPRESENTATION AND PROCESSING OF VISUAL INFORMATION , 2009 .

[43]  D. Ostry,et al.  Simultaneous Acquisition of Multiple Auditory–Motor Transformations in Speech , 2011, The Journal of Neuroscience.

[44]  Ewen N. MacDonald,et al.  Compensations in response to real-time formant perturbations of different magnitudes. , 2010, The Journal of the Acoustical Society of America.

[45]  HighWire Press The journal of neuroscience : the official journal of the Society for Neuroscience. , 1981 .

[46]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[47]  D. Wolpert,et al.  Internal models in the cerebellum , 1998, Trends in Cognitive Sciences.

[48]  Sophie K. Scott,et al.  The functional neuroanatomy of prelexical processing in speech perception , 2004, Cognition.

[49]  Daniel R. Lametti,et al.  Sensory Preference in Speech Production Revealed by Simultaneous Alteration of Auditory and Somatosensory Feedback , 2012, The Journal of Neuroscience.

[50]  E. Formisano,et al.  Neural correlates of verbal feedback processing: An fMRI study employing overt speech , 2007, Human brain mapping.

[51]  S. Sober,et al.  Vocal learning is constrained by the statistics of sensorimotor experience , 2012, Proceedings of the National Academy of Sciences.

[52]  Jean-Luc Schwartz,et al.  The Complementary Roles of Auditory and Motor Information Evaluated in a Bayesian Perceptuo-Motor Model of Speech Perception , 2017, Psychological review.

[53]  Feature analysis of Swedish vowels - a revisit , 2007 .

[54]  K. Munhall,et al.  Compensation following real-time manipulation of formants in isolated vowels. , 2006, The Journal of the Acoustical Society of America.

[55]  David J. Ostry,et al.  Speech Motor Learning in Profoundly Deaf Adults , 2008, Nature Neuroscience.

[56]  Sethu Vijayakumar,et al.  Unifying the Sensory and Motor Components of Sensorimotor Adaptation , 2008, NIPS.

[57]  Anatol G. Feldman,et al.  Threshold position control of arm movement with anticipatory increase in grip force , 2007, Experimental Brain Research.

[58]  P. McGuire,et al.  An fMRI study of verbal self-monitoring: neural correlates of auditory verbal feedback. , 2006, Cerebral cortex.

[59]  D J Ostry,et al.  A dynamic biomechanical model for neural control of speech production. , 1998, The Journal of the Acoustical Society of America.

[60]  Clément Moulin-Frier,et al.  COSMO ("Communicating about Objects using Sensory-Motor Operations"): A Bayesian modeling framework for studying speech communication and the emergence of phonological systems , 2015, J. Phonetics.

[61]  David J. Ostry,et al.  Auditory plasticity and speech motor learning , 2009, Proceedings of the National Academy of Sciences.

[62]  Pascal Perrier,et al.  Compensation strategies for the perturbation of the rounded vowel [u] using a lip-tube : A study of the control space in speech production , 1995 .

[63]  David J. Ostry,et al.  Somatosensory function in speech perception , 2009, Proceedings of the National Academy of Sciences.

[64]  T. Florian Jaeger,et al.  A Bayesian Belief Updating Model of Phonetic Recalibration and Selective Adaptation , 2011, CMCL@ACL.

[65]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[66]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[67]  Richard H R Hahnloser,et al.  A Bayesian Account of Vocal Adaptation to Pitch-Shifted Auditory Feedback , 2017, PloS one.

[68]  J. Perkell,et al.  Sensorimotor adaptation to feedback perturbations of vowel acoustics and its relation to perception. , 2007, The Journal of the Acoustical Society of America.

[69]  Satrajit S. Ghosh,et al.  Mechanisms of Vowel Production: Auditory Goals and Speaker Acuity , 2008 .

[70]  David J Ostry,et al.  Modifiability of generalization in dynamics learning. , 2007, Journal of neurophysiology.

[71]  S. Nagarajan,et al.  Sensorimotor adaptation affects perceptual compensation for coarticulation. , 2017, The Journal of the Acoustical Society of America.

[72]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[73]  J. Schwartz,et al.  The Perception-for-Action-Control Theory (PACT): A perceptuo-motor theory of speech perception , 2012, Journal of Neurolinguistics.

[74]  Julien Diard,et al.  Optimal speech motor control and token-to-token variability: a Bayesian modeling approach , 2015, Biological Cybernetics.

[75]  R. Jacobs,et al.  WITHIN CATEGORY PHONETIC VARIABILITY AFFECTS PERCEPTUAL UNCERTAINTY , 2007 .

[76]  D. Ostry,et al.  Somatosensory basis of speech production , 2003, Nature.

[77]  David J Ostry,et al.  Specificity of Speech Motor Learning , 2008, The Journal of Neuroscience.

[78]  David J. Ostry,et al.  A critical evaluation of the force control hypothesis in motor control , 2003, Experimental Brain Research.

[79]  D. Ostry,et al.  Nonhomogeneous transfer reveals specificity in speech motor learning. , 2012, Journal of neurophysiology.

[80]  B. Atal,et al.  Inversion of articulatory-to-acoustic transformation in the vocal tract by a computer-sorting technique. , 1978, The Journal of the Acoustical Society of America.

[81]  Keith Johnson,et al.  Partial Compensation for Altered Auditory Feedback: A Tradeoff with Somatosensory Feedback? , 2012, Language and speech.

[82]  Ewen N. MacDonald,et al.  Probing the independence of formant control using altered auditory feedback. , 2011, The Journal of the Acoustical Society of America.