Cross-modal speech perception in adults and infants using nonspeech auditory stimuli.

Adults and infants were tested for the capacity to detect correspondences between nonspeech sounds and real vowels. The /i/ and /a/ vowels were presented in 3 different ways: auditory speech, silent visual faces articulating the vowels, or mentally imagined vowels. The nonspeech sounds were either pure tones or 3-tone complexes that isolated a single feature of the vowel without allowing the vowel to be identified. Adults perceived an orderly relation between the nonspeech sounds and vowels. They matched high-pitched nonspeech sounds to /i/ vowels and low-pitched nonspeech sounds to /a/ vowels. In contrast, infants could not match nonspeech sounds to the visually presented vowels. Infants' detection of correspondence between auditory and visual speech appears to require the whole speech signal; with development, an isolated feature of the vowel is sufficient for detection of the cross-modal correspondence.

[1]  Norman J. Lass,et al.  Speech and Language: Advances in Basic Research and Practice , 1979 .

[2]  D. Gentner.  Metaphor as Structure Mapping: The Relational Shift. , 1988 .

[3]  千葉 勉,et al.  The vowel : its nature and structure , 1941 .

[4]  D. Pisoni,et al.  Perception of the duration of rapid spectrum changes in speech and nonspeech signals , 1983, Perception & psychophysics.

[5]  David Taylor.  Hearing by Eye: The Psychology of Lip-Reading , 1988 .

[6]  D W Massaro,et al.  American Psychological Association, Inc. Evaluation and Integration of Visual and Auditory Information in Speech Perception , 2022 .

[7]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[8]  J. D. Miller,et al.  Auditory-perceptual interpretation of the vowel. , 1989, The Journal of the Acoustical Society of America.

[9]  G. Fant,et al.  Auditory analysis and perception of speech , 1975 .

[10]  R E Remez,et al.  Perceptual normalization of vowels produced by sinusoidal voices. , 1987, Journal of experimental psychology. Human perception and performance.

[11]  D. Massaro.  Speech Perception By Ear and Eye: A Paradigm for Psychological Inquiry , 1989 .

[12]  H. S. Gopal,et al.  A perceptual model of vowel recognition based on the auditory representation of American English vowels. , 1986, The Journal of the Acoustical Society of America.

[13]  Gunnar Fant,et al.  Speech sounds and features , 1973 .

[14]  H. Traunmüller.  Perceptual dimension of openness in vowels. , 1981, The Journal of the Acoustical Society of America.

[15]  P. Kuhl.  Speech perception in early infancy: perceptual constancy for spectrally dissimilar vowel categories. , 1979, The Journal of the Acoustical Society of America.

[16]  D. Pisoni,et al.  Speech perception without traditional speech cues. , 1981, Science.

[17]  R. Diehl,et al.  An auditory basis for the stimulus-length effect in the perception of stops and glides. , 1989, The Journal of the Acoustical Society of America.

[18]  A M Liberman,et al.  Perception of the speech code. , 1967, Psychological review.

[19]  R. Plomp.  Pitch of complex tones. , 1966, The Journal of the Acoustical Society of America.

[20]  L. Chistovich,et al.  The ‘center of gravity’ effect in vowel spectra and critical distance between the formants: Psychoacoustical study of the perception of vowel-like stimuli , 1979, Hearing Research.

[21]  P E Rubin,et al.  On the perception of intonation from sinusoidal sentences , 1984, Perception & psychophysics.

[22]  R. Campbell,et al.  Hearing by eye : the psychology of lip-reading , 1988 .

[23]  R E Pastore,et al.  Discrimination and labeling of noise-buzz sequences with varying noise-lead times: an example of categorical perception. , 1976, The Journal of the Acoustical Society of America.

[24]  M. Halle,et al.  Preliminaries to Speech Analysis: The Distinctive Features and Their Correlates , 1961 .

[25]  P. Kuhl.  Perception of auditory equivalence classes for speech in early infancy , 1983 .

[26]  A. Meltzoff,et al.  The bimodal perception of speech in infancy. , 1982, Science.

[27]  P. Ladefoged.  Three areas of experimental phonetics , 1967 .

[28]  C A Fowler,et al.  Sound-producing sources as objects of perception: rate normalization and nonspeech perception. , 1990, The Journal of the Acoustical Society of America.

[29]  A. Meltzoff,et al.  The Intermodal Representation of Speech in Infants , 1984 .

[30]  P. Farnsworth.  An Approach to the Study of Vocal Resonance , 1937 .

[31]  P K Kuhl,et al.  The role of visual information in the processing of , 1989, Perception & psychophysics.

[32]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[33]  B. Repp.  Categorical Perception: Issues, Methods, Findings , 1984 .

[34]  Q Summerfield,et al.  Use of Visual Information for Phonetic Perception , 1979, Phonetica.

[35]  Albert Yonas,et al.  Perceptual development in infancy: Minnesota Symposia on Child Psychology , 1988 .

[36]  J. Mullennix,et al.  Integral processing of phonemes: evidence for a phonetic mode of perception. , 1987, The Journal of the Acoustical Society of America.

[37]  B. Rosner,et al.  Categories and boundaries in speech and music* , 1974 .

[38]  M. Bornstein,et al.  Development in Infancy , 1982 .

[39]  D. Pisoni.  Identification and discrimination of the relative onset time of two component tones: implications for voicing perception in stops. , 1977, The Journal of the Acoustical Society of America.

[40]  M Studdert-Kennedy,et al.  Infant intermodal speech perception is a left-hemisphere function. , 1983, Science.

[41]  Q. Summerfield.  Some preliminaries to a comprehensive account of audio-visual speech perception. , 1987 .

[42]  R. Ritsma.  Frequencies dominant in the perception of the pitch of complex sounds. , 1966, The Journal of the Acoustical Society of America.

[43]  D B Pisoni,et al.  Discimination of relative onset time of two-component tones by infants. , 1980, The Journal of the Acoustical Society of America.

[44]  D. G. MacKay,et al.  Metaphor and Thought , 1980 .

[45]  S. Harnad.  Categorical Perception: The Groundwork of Cognition , 1990 .

[46]  H. Helmholtz,et al.  Book Reviews: On the Sensations of Tone as a Physiological Basis for the Theory of Music , 1954 .

[47]  A M Liberman,et al.  A specialization for speech perception. , 1989, Science.

[48]  P. Kuhl,et al.  Integral processing of visual place and auditory voicing information during phonetic perception. , 1991, Journal of experimental psychology. Human perception and performance.

[49]  P. D. Eimas,et al.  Speech Perception in Infants , 1971, Science.

[50]  Catherine T. Best,et al.  Discovering phonetic coherence in acoustic patterns , 1989, Perception & psychophysics.

[51]  Patricia K. Kuhl,et al.  The special-mechanisms debate in speech research: Categorization tests on animals and infants. , 1987 .

[52]  G. Fant,et al.  Two-formant Models, Pitch and Vowel Perception , 1975 .

[53]  H. Helmholtz,et al.  On the Sensations of Tone as a Physiological Basis for the Theory of Music , 2005 .