The shadow of a doubt? Evidence for perceptuo-motor linkage during auditory and audiovisual close-shadowing

One classical argument in favor of a functional role of the motor system in speech perception comes from the close-shadowing task in which a subject has to identify and to repeat as quickly as possible an auditory speech stimulus. The fact that close-shadowing can occur very rapidly and much faster than manual identification of the speech target is taken to suggest that perceptually induced speech representations are already shaped in a motor-compatible format. Another argument is provided by audiovisual interactions often interpreted as referring to a multisensory-motor framework. In this study, we attempted to combine these two paradigms by testing whether the visual modality could speed motor response in a close-shadowing task. To this aim, both oral and manual responses were evaluated during the perception of auditory and audiovisual speech stimuli, clear or embedded in white noise. Overall, oral responses were faster than manual ones, but it also appeared that they were less accurate in noise, which suggests that motor representations evoked by the speech input could be rough at a first processing stage. In the presence of acoustic noise, the audiovisual modality led to both faster and more accurate responses than the auditory modality. No interaction was however, observed between modality and response. Altogether, these results are interpreted within a two-stage sensory-motor framework, in which the auditory and visual streams are integrated together and with internally generated motor representations before a final decision may be available.

[1]  Bir Bhanu,et al.  Analysis-by-synthesis: Pedestrian tracking with crowd simulation models in a multi-camera video network , 2015, Comput. Vis. Image Underst..

[2]  Laurent Girin,et al.  A mediating role of the auditory dorsal pathway in selective adaptation to speech: A state-dependent transcranial magnetic stimulation study , 2013, Brain Research.

[3]  J. Schwartz,et al.  The Perception-for-Action-Control Theory (PACT): A perceptuo-motor theory of speech perception , 2012, Journal of Neurolinguistics.

[4]  Jean-Luc Schwartz,et al.  Adverse conditions improve distinguishability of auditory, motor, and perceptuo-motor theories of speech perception: An exploratory Bayesian modelling study , 2012 .

[5]  Luciano Fadiga,et al.  The role of the motor system in discriminating normal and degraded speech sounds , 2012, Cortex.

[6]  K. Watkins,et al.  Using TMS to study the role of the articulatory motor system in speech perception , 2011, Aphasiology.

[7]  Alessandro D’Ausilio,et al.  Tongue corticospinal modulation during attended verbal stimuli: Priming and coarticulation effects , 2011, Neuropsychologia.

[8]  T. Ehrenfeld Reflections on Mirror Neurons , 2011 .

[9]  Feng Rong,et al.  Sensorimotor Integration in Speech Processing: Computational Basis and Neural Organization , 2011, Neuron.

[10]  David Poeppel,et al.  Analysis by Synthesis: A (Re-)Emerging Program of Research for Language and Vision , 2010, Biolinguistics.

[11]  J. Schwartz,et al.  Shared and distinct neural correlates of vowel perception and production , 2010, Journal of Neurolinguistics.

[12]  Vincent L. Gracco,et al.  A mediating role of the premotor cortex in phoneme segmentation , 2009, Brain and Language.

[13]  K. Watkins,et al.  Motor Representations of Articulators Contribute to Categorical Perception of Speech Sounds , 2009, The Journal of Neuroscience.

[14]  L. Fadiga,et al.  The Motor Somatotopy of Speech Perception , 2009, Current Biology.

[15]  Stuart Rosen,et al.  The neural processing of masked speech: evidence for different mechanisms in the left and right temporal lobes. , 2009, The Journal of the Acoustical Society of America.

[16]  Lori L. Holt,et al.  Reflections on mirror neurons and speech perception , 2009, Trends in Cognitive Sciences.

[17]  Marco Iacoboni,et al.  The Essential Role of Premotor Cortex in Speech Perception , 2007, Current Biology.

[18]  Jeremy I. Skipper,et al.  Seeing Voices : How Cortical Areas Supporting Speech Production Mediate Audiovisual Speech Perception , 2007 .

[19]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[20]  Marco Iacoboni,et al.  Neural responses to non-native phonemes varying in producibility: Evidence for the sensorimotor nature of speech perception , 2006, NeuroImage.

[21]  Adriana A. Zekveld,et al.  Top–down and bottom–up processes in speech comprehension , 2006, NeuroImage.

[22]  M. Turvey,et al.  The motor theory of speech perception reviewed , 2006, Psychonomic bulletin & review.

[23]  Mikko Sams,et al.  Viewing speech modulates activity in the left SI mouth cortex , 2005, NeuroImage.

[24]  M. Iacoboni,et al.  Listening to speech activates motor areas involved in speech production , 2004, Nature Neuroscience.

[25]  Jeffery A. Jones,et al.  Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory–auditory/orosensory internal models , 2004, NeuroImage.

[26]  David A. Medler,et al.  Neural correlates of sensory and decision processes in auditory object identification , 2004, Nature Neuroscience.

[27]  C. Fowler,et al.  Rapid access to speech gestures in perception: Evidence from choice and simple response time tasks. , 2003, Journal of memory and language.

[28]  J. Kalinowski,et al.  Choral speech: the amelioration of stuttering via imitation and the mirror neuronal system , 2003, Neuroscience & Biobehavioral Reviews.

[29]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[30]  G. Aschersleben,et al.  The Theory of Event Coding (TEC): a framework for perception and action planning. , 2001, The Behavioral and brain sciences.

[31]  Jeesun Kim,et al.  Repeating and Remembering Foreign Language Words: Implications for Language Teaching Systems , 2001, Artificial Intelligence Review.

[32]  W. Prinz Perception and Action Planning , 1997 .

[33]  E. Tobey,et al.  Shadowed and simple reaction times in stutterers and nonstutterers. , 1989, The Journal of the Acoustical Society of America.

[34]  David Taylor Hearing by Eye: The Psychology of Lip-Reading , 1988 .

[35]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[36]  Kam L. Wong Analysis or synthesis , 1985 .

[37]  F. X. Castellanos,et al.  Speech-production measures of speech perception: rapid shadowing of VCV syllables. , 1980, The Journal of the Acoustical Society of America.

[38]  S. Blumstein,et al.  Invariant cues for place of articulation in stop consonants. , 1978, The Journal of the Acoustical Society of America.

[39]  R. Brubaker Models for the perception of speech and visual form: Weiant Wathen-Dunn, ed.: Cambridge, Mass., The M.I.T. Press, I–X, 470 pages , 1968 .

[40]  P. Denes On the Motor Theory of Speech Perception , 1965 .

[41]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[42]  J. C. Cotton NORMAL "VISUAL HEARING". , 1935, Science.

[43]  R. Diehl,et al.  Speech Perception , 2004, Annual review of psychology.

[44]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[45]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[46]  Björn Lindblom,et al.  Explaining Phonetic Variation: A Sketch of the H&H Theory , 1990 .

[47]  C. Fowler An event approach to the study of speech perception from a direct realist perspective , 1986 .

[48]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .