The perception of anticipatory labial coarticulation by blind listeners in noise: A comparison with sighted listeners in audio-only, visual-only and audiovisual conditions

Abstract This study investigates the time course of the perception of the /i-y/ contrast by French-speaking blind listeners using a gating paradigm. The performances of the blind listeners in discrimination and identification are compared with the range of performances exhibited by sighted perceivers when stimuli are presented auditorily, visually and audiovisually, whether in acoustically non degraded or in noisy conditions. Results provide evidence in favor of partial compensation for visual deprivation in speech perception. Blind listeners outperformed sighted participants in discriminating between auditorily-presented gated stimuli, particularly in noisy conditions. But this small advantage allowed them to compensate only partially for their inability to exploit visual information in order to process coarticulated speech as quickly and efficiently as sighted controls.

[1]  C Muchnik,et al.  Central auditory skills in blind and sighted subjects. , 1991, Scandinavian audiology.

[2]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[3]  Lawrence D. Rosenblum,et al.  Influences of selective adaptation on perception of audiovisual speech , 2016, J. Phonetics.

[4]  Kenneth Hugdahl,et al.  Blind individuals show enhanced perceptual and attentional sensitivity for identification of speech sounds. , 2004, Brain research. Cognitive brain research.

[5]  F. Rösler,et al.  Speech processing activates visual cortex in congenitally blind humans , 2002, The European journal of neuroscience.

[6]  J. Schwartz.  A reanalysis of McGurk data suggests that audiovisual fusion in speech perception is subject-dependent. , 2010, The Journal of the Acoustical Society of America.

[7]  C. Spence,et al.  Auditory, tactile, and audiotactile information processing following visual deprivation. , 2013, Psychological bulletin.

[8]  J. Peelle,et al.  Prediction and constraint in audiovisual speech perception , 2015, Cortex.

[9]  Asif A. Ghazanfar,et al.  The Natural Statistics of Audiovisual Speech , 2009, PLoS Comput. Biol..

[10]  D. Massaro,et al.  The temporal distribution of information in audiovisual spoken-word identification , 2010, Attention, perception & psychophysics.

[11]  Andrew Faulkner,et al.  AUDITORY-VISUAL L 2 SPEECH PERCEPTION CUES AND ACOUSTIC-PHONETIC CONTEXT FO OF ENGLISH , 2001 .

[12]  Auditory-visual speech perception in older people: The effect of visual acuity , 2004 .

[13]  Victor Candas,et al.  Enhanced sensitivity to echo cues in blind subjects , 2005, Experimental Brain Research.

[14]  J. Werker,et al.  An exploration of why preschoolers perform differently than do adults in audiovisual speech perception tasks. , 1997, Journal of experimental child psychology.

[15]  R. Sock,et al.  Etude préliminaire de la perception précoce des voyelles labialisées par des auditeurs déficients visuels , 2010 .

[16]  P F Seitz,et al.  The use of visible speech cues for improving auditory detection of spoken sentences. , 2000, The Journal of the Acoustical Society of America.

[17]  D. Pisoni,et al.  A Longitudinal Study of Audiovisual Speech Perception by Children with Hearing Loss Who have Cochlear Implants. , 2003, The Volta review.

[18]  Chip Gerfen,et al.  Audiovisual perceptual learning with multiple speakers , 2016, J. Phonetics.

[19]  Roy H. Hamilton,et al.  Absolute pitch in blind musicians , 2004, Neuroreport.

[20]  M. Schouten,et al.  Categorical perception depends on the discrimination task , 2004, Perception & psychophysics.

[21]  J. Schwartz,et al.  Seeing to hear better: evidence for early audio-visual interactions in speech identification , 2004, Cognition.

[22]  R. Zatorre,et al.  Voice perception in blind persons: A functional magnetic resonance imaging study , 2009, Neuropsychologia.

[23]  K. Tiippana.  What is the McGurk effect? , 2014, Front. Psychol..

[24]  H. Burton,et al.  Dissociating cortical regions activated by semantic and phonological tasks: a FMRI study in blind and sighted people. , 2003, Journal of neurophysiology.

[25]  Eva Reinisch,et al.  Exposure modality, input variability and the categories of perceptual recalibration , 2016, J. Phonetics.

[26]  Lawrence D Rosenblum,et al.  Speech Perception as a Multimodal Phenomenon , 2008, Current directions in psychological science.

[27]  Ingrid S Johnsrude,et al.  The role of visual speech information in supporting perceptual learning of degraded speech. , 2012, Journal of experimental psychology. Applied.

[28]  H. Ackermann,et al.  Training of ultra-fast speech comprehension induces functional reorganization of the central-visual system in late-blind humans , 2013, Front. Hum. Neurosci..

[29]  J Robert-Ribes,et al.  Complementarity and synergy in bimodal speech: auditory, visual, and audio-visual identification of French oral vowels in noise. , 1998, The Journal of the Acoustical Society of America.

[30]  M. T. Keurs,et al.  Voice Identification of Prepubertal Boys and Girls by Normally Sighted and Visually Handicapped Subjects , 1987, Language and speech.

[31]  Alan Wee-Chung Liew,et al.  Visual Speech Recognition: Lip Segmentation and Mapping , 2008 .

[32]  L. Bernstein,et al.  Audiovisual Speech Binding: Convergence or Association? , 2004 .

[33]  E. Winograd,et al.  Voice recognition: effects of orienting task, and a test of blind versus sighted listeners. , 1984, The American journal of psychology.

[34]  Yue Wang,et al.  Influence of native language phonetic system on audio-visual speech perception , 2009, J. Phonetics.

[35]  R. Saxe,et al.  Language processing in the occipital cortex of congenitally blind adults , 2011, Proceedings of the National Academy of Sciences.

[36]  R. Zatorre,et al.  A Functional Neuroimaging Study of Sound Localization: Visual Cortex Activity Predicts Performance in Early-Blind Individuals , 2005, PLoS biology.

[37]  Takaaki Kuratate,et al.  Linking facial animation, head motion and speech acoustics , 2002, J. Phonetics.

[38]  J. Schwartz,et al.  The Dispersion-Focalization Theory of vowel systems , 1997 .

[39]  W Niemeyer,et al.  Do the blind hear better? Investigations on auditory processing in congenital or early acquired blindness. II. Central functions. , 1981, Audiology : official organ of the International Society of Audiology.

[40]  Colline Poirier,et al.  Auditory motion perception activates visual motion areas in early blind subjects , 2006, NeuroImage.

[41]  Martijn Baart,et al.  Quantifying lip-read-induced suppression and facilitation of the auditory N1 and P2 reveals peak enhancements and delays. , 2016, Psychophysiology.

[42]  J. Roy.  Anticipation labiale et perception visuelle , 2012 .

[43]  Hartmut Traunmüller,et al.  Audiovisual perception of openness and lip rounding in front vowels , 2007, J. Phonetics.

[44]  Trevor H. Chen,et al.  Seeing pitch: visual information for lexical tones of Mandarin-Chinese. , 2008, The Journal of the Acoustical Society of America.

[45]  M. P. Zwiers,et al.  A Spatial Hearing Deficit in Early-Blind Humans , 2001, The Journal of Neuroscience.

[46]  B. Dodd,et al.  Review of visual speech perception by hearing and hearing-impaired people: clinical implications. , 2009, International journal of language & communication disorders.

[47]  David C. Reutens,et al.  Early but not late-blindness leads to enhanced auditory perception , 2010, Neuropsychologia.

[48]  M. Wallace,et al.  Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. , 2012, Journal of experimental psychology. Human perception and performance.

[49]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[50]  Jeffery A. Jones,et al.  Visual Prosody and Speech Intelligibility , 2004, Psychological science.

[51]  F. Rösler,et al.  Event-related potentials during auditory and somatosensory discrimination in sighted and blind human subjects. , 1996, Brain research. Cognitive brain research.

[52]  K. Sekiyama,et al.  Enhanced audiovisual integration with aging in speech perception: a heightened McGurk effect in older adults , 2014, Front. Psychol..

[53]  R. Saxe,et al.  A sensitive period for language in the visual cortex: Distinct patterns of plasticity in congenitally versus late blind adults , 2011, Brain and Language.

[54]  J. Navarra,et al.  Hearing lips in a second language: visual articulatory information enables the perception of second language sounds , 2007, Psychological research.

[55]  B. Clifford,et al.  The Voice-Recognition Accuracy of Blind Listeners , 1983, Perception.

[56]  Marc Sato,et al.  Auditory-tactile speech perception in congenitally blind and sighted adults , 2010, Neuropsychologia.

[57]  S. Dupont,et al.  Production and perception of French vowels by congenitally blind adults and sighted adults. , 2009, The Journal of the Acoustical Society of America.

[58]  B. Gick,et al.  Tactile enhancement of auditory and visual speech perception in untrained perceivers. , 2008, The Journal of the Acoustical Society of America.

[59]  Andrew Faulkner,et al.  The use of visual cues in the perception of non-native consonant contrasts. , 2006, The Journal of the Acoustical Society of America.

[61]  C. Colin,et al.  Atypical audio-visual speech perception and McGurk effects in children with specific language impairment , 2014, Front. Psychol..

[62]  E. Zohary,et al.  Transcranial magnetic stimulation of the occipital pole interferes with verbal processing in blind subjects , 2004, Nature Neuroscience.

[63]  Jürgen Trouvain,et al.  On the comprehension of extremely fast synthetic speech , 2007 .

[64]  S. Hillyard,et al.  Improved auditory spatial tuning in blind humans , 1999, Nature.

[65]  D. Lewkowicz,et al.  Recognition of amodal language identity emerges in infancy , 2013, International journal of behavioral development.

[66]  Doğu Erdener.  Basic to applied research: the benefits of audio-visual speech perception research in teaching foreign languages , 2016 .

[67]  William J. Idsardi,et al.  The use of auditory and visual context in speech perception by listeners with normal hearing and listeners with cochlear implants , 2013, Front. Psychol..

[68]  Christian Abry,et al.  Speech face perception is locked to anticipation in speech production , 2010, Speech Commun..

[69]  Sandra Gordon-Salant,et al.  Recognition of rapid speech by blind and sighted older adults. , 2011, Journal of speech, language, and hearing research : JSLHR.

[70]  Marie-Agnès Cathiard,et al.  La perception visuelle de l'anticipation des gestes vocaliques : cohérence des évènements audibles et visibles dans le flux de la parole , 1994 .

[71]  A. Simon,et al.  La prononciation du français en Belgique , 2009 .

[72]  A. Braun.  Speaker-recognition ability of blind and sighted subjects , 2012 .

[73]  Jean-Luc Schwartz,et al.  The shadow of a doubt? Evidence for perceptuo-motor linkage during auditory and audiovisual close-shadowing , 2014, Front. Psychol..

[74]  H. Smeds.  Blindness and Second Language Acquisition : Studies of Cognitive Advantages in Blind L1 and L2 speakers , 2015 .

[75]  Paul Boersma,et al.  Praat, a system for doing phonetics by computer , 2002 .

[76]  David Whitney,et al.  Ultrafine spatial acuity of blind expert human echolocators , 2012, Experimental Brain Research.

[78]  Tony Charman,et al.  Atypical Audiovisual Speech Integration in Infants at Risk for Autism , 2012, PloS one.

[79]  P. Bertelson,et al.  Visual Recalibration of Auditory Speech Identification , 2003, Psychological science.

[80]  Jean-Luc Schwartz,et al.  No, There Is No 150 ms Lead of Visual Speech on Auditory Speech, but a Range of Audiovisual Asynchronies Varying from Small Audio Lead to Large Audio Lag , 2014, PLoS Comput. Biol..