The use of visual cues in the perception of non-native consonant contrasts.

This study assessed the extent to which second-language learners are sensitive to phonetic information contained in visual cues when identifying a non-native phonemic contrast. In experiment 1, Spanish and Japanese learners of English were tested on their perception of a labial/ labiodental consonant contrast in audio (A), visual (V), and audio-visual (AV) modalities. Spanish students showed better performance overall, and much greater sensitivity to visual cues than Japanese students. Both learner groups achieved higher scores in the AV than in the A test condition, thus showing evidence of audio-visual benefit. Experiment 2 examined the perception of the less visually-salient /1/-/r/ contrast in Japanese and Korean learners of English. Korean learners obtained much higher scores in auditory and audio-visual conditions than in the visual condition, while Japanese learners generally performed poorly in both modalities. Neither group showed evidence of audio-visual benefit. These results show the impact of the language background of the learner and visual salience of the contrast on the use of visual cues for a non-native contrast. Significant correlations between scores in the auditory and visual conditions suggest that increasing auditory proficiency in identifying a non-native contrast is linked with an increasing proficiency in using visual cues to the contrast.

[1]  G. McRoberts,et al.  Infant Perception of Non-Native Consonant Contrasts that Adults Assimilate in Different Ways , 2003, Language and speech.

[2]  Debra M. Hardison,et al.  Bimodal Speech Perception by Native and Nonnative Speakers of English: Factors Influencing the McGurk Effect , 1996 .

[3]  Randy L. Diehl,et al.  MAPPING THE TASK FOR THE SECOND LANGUAGE LEARNER: THE CASE OF JAPANESE ACQUISITION OF /R/ AND /L/ , 2004 .

[4]  K. Grant,et al.  Measures of auditory-visual integration in nonsense syllables and sentences. , 1998, The Journal of the Acoustical Society of America.

[5]  D. Pisoni,et al.  Training Japanese listeners to identify English /r/ and /l/: IV. Some effects of perceptual learning on speech production. , 1997, The Journal of the Acoustical Society of America.

[6]  Debra M. Hardison,et al.  Variability in bimodal spoken language processing by native and nonnative speakers of English: A closer look at effects of speech style , 2005, Speech Commun..

[7]  J. Werker,et al.  Matching phonetic information in lips and voice is robust in 4.5-month-old infants , 1999 .

[8]  Paul Iverson,et al.  Phonetic training with acoustic cue manipulations: a comparison of methods for teaching English /r/-/l/ to Japanese adults. , 2005, The Journal of the Acoustical Society of America.

[9]  K. Sekiyama,et al.  Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects , 1997, Perception & psychophysics.

[10]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[11]  Kaoru Sekiyama,et al.  A few factors which affect the degree of incorporating lip-read information into speech perception , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[12]  Andrew Faulkner,et al.  Effect of audiovisual perceptual training on the perception and production of consonants by Japanese learners of English , 2005, Speech Commun..

[13]  A. Meltzoff,et al.  The bimodal perception of speech in infancy. , 1982, Science.

[14]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[15]  A A Montgomery,et al.  Auditory and visual contributions to the perception of consonants. , 1974, Journal of speech and hearing research.

[16]  Ronald A. Cole,et al.  TOOLS FOR RESEARCH AND EDUCATION IN SPEECH SCIENCE , 1999 .

[17]  Valérie Hazan,et al.  The development of phonemic categorization in children aged 6-12 , 2000, J. Phonetics.

[18]  J. Werker,et al.  Two-month-old infants match phonetic information in lips and voice , 2003 .

[19]  C. Mayo,et al.  Adult-child differences in acoustic cue weighting are influenced by segmental context: children are not always perceptually biased toward transitions. , 2004, The Journal of the Acoustical Society of America.

[20]  James Emil Flege,et al.  Interaction between the native and second language phonetic subsystems , 2003, Speech Commun..

[21]  J C Ingram,et al.  Language, context, and speaker effects in the identification and discrimination of English /r/ and /l/ by Japanese and Korean listeners. , 1998, The Journal of the Acoustical Society of America.

[22]  J. Flege,et al.  Amount of native-language (L1) use affects the pronunciation of an L2 , 1997 .

[23]  M E Miller,et al.  Predicting developmental shifts in perceptual weighting schemes. , 1997, The Journal of the Acoustical Society of America.

[24]  P C Gordon,et al.  Ability in perceiving nonnative contrasts: Performance on natural and synthetic speech stimuli , 2001, Perception & psychophysics.

[25]  B. Walden,et al.  Effects of training on the visual recognition of consonants. , 1977, Journal of speech and hearing research.

[26]  Y. Tohkura,et al.  Inter-language differences in the influence of visual cues in speech perception. , 1993 .

[27]  C. Best.  A direct realist view of cross-language speech perception , 1995 .

[28]  Patricia K. Kohl.  Early linguistic experience and phonetic perception: implications for theories of developmental speech perception , 1993 .

[29]  C. Fowler.  An event approach to the study of speech perception from a direct realist perspective , 1986 .

[30]  Adrian Fourcin,et al.  An application of speech processing and encoding scheme for Chinese lexical tone and consonant perception by hearing impaired listeners , 1991, EUROSPEECH.

[31]  Antoinette T. Gesi,et al.  Bimodal speech perception: an examination across languages , 1993 .

[32]  Andrew Faulkner,et al.  AUDITORY-VISUAL L 2 SPEECH PERCEPTION CUES AND ACOUSTIC-PHONETIC CONTEXT FO OF ENGLISH , 2001 .

[33]  L A Thompson,et al.  Developmental changes in visual and auditory contributions to speech perception. , 1986, Journal of experimental child psychology.

[34]  Q. Summerfield.  Audio-visual Speech Perception, Lipreading and Artificial Stimulation , 1983 .

[35]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[36]  J. Werker,et al.  Cross-language speech perception: Evidence for perceptual reorganization during the first year of life , 1984 .

[37]  S. Rosen,et al.  Individual variability in the perception of cues to place contrasts in initial stops , 1991, Perception & psychophysics.

[38]  L. Lisker,et al.  Voice-Timing Perception in Spanish Word-Initial Stops. , 1973 .

[39]  D. Hardison.  Acquisition of second-language speech: Effects of visual cues, context, and talker variability , 2003, Applied Psycholinguistics.

[40]  Tsuneo Yamada,et al.  Perceived phonetic dissimilarity and L2 speech learning: the case of Japanese /r/ and English /l/ and /r/ , 2004, J. Phonetics.

[41]  Y. Tohkura,et al.  A perceptual interference account of acquisition difficulties for non-native phonemes , 2003, Cognition.

[42]  Kuniko Y. Nielsen.  Segmental differences in the visual contribution to speech inteligibility , 2004, INTERSPEECH.