Interleaved lexical and audiovisual information can retune phoneme boundaries

To adapt to situations in which speech perception is difficult, listeners can adjust boundaries between phoneme categories using perceptual learning. Such adjustments can draw on lexical information in surrounding speech, or on visual cues via speech-reading. In the present study, listeners proved they were able to flexibly adjust the boundary between two plosive/stop consonants, /p/-/t/, using both lexical and speech-reading information and given the same experimental design for both cue types. Videos of a speaker pronouncing pseudo-words and audio recordings of Dutch words were presented in alternating blocks of either stimulus type. Listeners were able to switch between cues to adjust phoneme boundaries, and resulting effects were comparable to results from listeners receiving only a single source of information. Overall, audiovisual cues (i.e., the videos) produced the stronger effects, commensurate with their applicability for adapting to noisy environments. Lexical cues were able to induce effects with fewer exposure stimuli and a changing phoneme bias, in a design unlike most prior studies of lexical retuning. While lexical retuning effects were relatively weaker compared to audiovisual recalibration, this discrepancy could reflect how lexical retuning may be more suitable for adapting to speakers than to environments. Nonetheless, the presence of the lexical retuning effects suggests that it may be invoked at a faster rate than previously seen. In general, this technique has further illuminated the robustness of adaptability in speech perception, and offers the potential to enable further comparisons across differing forms of perceptual learning.

[1]  J. McQueen,et al.  Perceptual learning in speech: Stability over time (L) , 2006 .

[2]  Holger Mitterer,et al.  Visual speech influences speech perception immediately but not automatically , 2017, Attention, perception & psychophysics.

[3]  A. Samuel,et al.  Perceptual adjustments to multiple speakers , 2007 .

[4]  W. Ganong Phonetic categorization in auditory word perception. , 1980, Journal of experimental psychology. Human perception and performance.

[5]  Jean Vroomen,et al.  Recalibration of Phonetic Categories by Lipread Speech: Measuring Aftereffects After a 24-hour Delay , 2009, Language and speech.

[6]  A G Samuel,et al.  Knowing a Word Affects the Fundamental Perception of The Sounds Within it , 2001, Psychological science.

[7]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[8]  Dominic W. Massaro,et al.  Audiovisual speech perception and word recognition , 2007 .

[9]  P. Bertelson,et al.  Visual Recalibration of Auditory Speech Identification , 2003, Psychological science.

[10]  P. Bertelson,et al.  Visual recalibration and selective adaptation in auditory–visual speech perception: Contrasting build-up courses , 2007, Neuropsychologia.

[11]  J. McQueen The influence of the lexicon on phonetic categorization: stimulus quality in word-final ambiguity. , 1991, Journal of experimental psychology. Human perception and performance.

[12]  B. Repp Perceptual equivalence of two kinds of ambiguous speech stimuli , 1981 .

[13]  Paul Boersma,et al.  Speak and unSpeak with P RAATRAAT , 2002 .

[14]  J. McQueen,et al.  Perceptual learning in speech: stability over time. , 2006, The Journal of the Acoustical Society of America.

[15]  Marc Brysbaert,et al.  WordGen: A tool for word selection and nonword generation in Dutch, English, German, and French , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[16]  Hwee Ling Lee Audiovisual speech perception , 2012 .

[17]  A. Samuel,et al.  Perceptual learning for speech , 2009, Attention, perception & psychophysics.

[18]  Jean Vroomen,et al.  A spatial gradient in phonetic recalibration by lipread speech , 2016, J. Phonetics.

[19]  Béatrice de Gelder,et al.  Selective adaptation and recalibration of auditory speech by lipread information: Dissipation , 2004, AVSP.

[20]  P. D. Eimas,et al.  Selective adaptation of linguistic feature detectors , 1973 .

[21]  Chip Gerfen,et al.  Audiovisual perceptual learning with multiple speakers , 2016, J. Phonetics.

[22]  A. Kappers,et al.  Tactile perception of thermal diffusivity , 2009, Attention, perception & psychophysics.

[23]  D. Norris,et al.  How abstract phonemic categories are necessary for coping with speaker-related variation , 2010 .

[24]  Jean Vroomen,et al.  Recalibration of phonetic categories by lipread speech versus lexical information. , 2007, Journal of experimental psychology. Human perception and performance.

[25]  Jean Vroomen,et al.  Phonetic recalibration in audiovisual speech , 2012 .

[26]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[27]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[28]  Jean Vroomen,et al.  Recalibration of auditory phonemes by lipread speech is ear-specific , 2015, Cognition.

[29]  A. Cutler,et al.  No L1 privilege in talker adaptation , 2019, Bilingualism: Language and Cognition.

[30]  Dave F. Kleinschmidt,et al.  Robust speech perception: recognize the familiar, generalize to the similar, and adapt to the novel. , 2015, Psychological review.

[31]  Anne Cutler,et al.  Native listening: The flexibility dimension , 2012 .

[32]  F. D. de Lange,et al.  Rapid recalibration of speech perception after experiencing the McGurk illusion , 2018, Royal Society Open Science.