Lipreading and audio-visual speech perception.

This paper reviews progress in understanding the psychology of lipreading and audio-visual speech perception. It considers four questions. What distinguishes better from poorer lipreaders? What are the effects of introducing a delay between the acoustical and optical speech signals? What have attempts to produce computer animations of talking faces contributed to our understanding of the visual cues that distinguish consonants and vowels? Finally, how should the process of audio-visual integration in speech perception be described; that is, how are the sights and sounds of talking faces represented at their conflux?

[1]  L. Braida.  Crossmodal Integration in the Identification of Consonant Segments , 1991, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[2]  William D Marslen-Wilson,et al.  Processing interactions and lexical access during word recognition in continuous speech , 1978, Cognitive Psychology.

[3]  A. Macleod,et al.  LIPS, TEETH, AND THE BENEFITS OF LIPREADING , 1989 .

[4]  V J Samar,et al.  Visual evoked-response correlates of speechreading performance in normal-hearing adults: a replication and factor analytic extension. , 1983, Journal of speech and hearing research.

[5]  C. Fowler.  The perception of phonetic gestures , 1991 .

[6]  E. Spelke.  Infants' intermodal perception of events , 1976, Cognitive Psychology.

[7]  Dennis H. Klatt,et al.  Speech perception: a model of acoustic–phonetic analysis and lexical access , 1979 .

[8]  B. Walden,et al.  Effects of training on the visual recognition of consonants. , 1977, Journal of speech and hearing research.

[9]  J. Rönnberg,et al.  Information-processing skill and speech-reading. , 1989, British journal of audiology.

[10]  A. Duquesnoy.  The intelligibility of sentences in quiet and in noise in aged listeners. , 1983, The Journal of the Acoustical Society of America.

[11]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[12]  Q. Summerfield.  Audio-visual Speech Perception, Lipreading and Artificial Stimulation , 1983 .

[13]  A. Meltzoff,et al.  Integrating speech information across talkers, gender, and sensory modality: Female faces and male voices in the McGurk effect , 1991, Perception & psychophysics.

[14]  R Plomp,et al.  The effect of speechreading on the speech-reception threshold of sentences in noise. , 1987, The Journal of the Acoustical Society of America.

[15]  N. M. Brooke,et al.  Analysis, synthesis, and perception of visible articulatory movements , 1983 .

[16]  L. Bahrick,et al.  Infants' Bimodal Perception of Gender , 1991 .

[17]  A. M. Mimpen,et al.  Improving the reliability of testing the speech reception threshold for sentences. , 1979, Audiology : official organ of the International Society of Audiology.

[18]  B. Walden,et al.  Perception of synthetic visual consonant-vowel articulations. , 1987, Journal of speech and hearing research.

[19]  Q Summerfield.  Speech perception in normal and impaired hearing. , 1987, British medical bulletin.

[20]  A. W. Ellis,et al.  NEURO-COGNITIVE PROCESSING OF FACES AND VOICES , 1989 .

[21]  C A Binnie.  Attitude changes following speechreading training. , 1977, Scandinavian audiology.

[22]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[23]  D C Shepherd,et al.  Visual-neural correlate of speechreading ability in normal-hearing adults: reliability. , 1982, Journal of speech and hearing research.

[24]  D. Brooks.  Lip-reading instruction and hearing aid use. , 1989, British journal of audiology.

[25]  R Plomp,et al.  Auditive and cognitive factors in speech perception by elderly listeners. I: Development of test battery. , 1989, The Journal of the Acoustical Society of America.

[26]  K. Stevens,et al.  Development of a Quantitative Description of Vowel Articulation , 1955 .

[27]  S. Arlinger,et al.  Visual evoked potentials: relation to adult speechreading and cognitive function. , 1989, Journal of speech and hearing research.

[28]  D H Klatt,et al.  Review of text-to-speech conversion for English. , 1987, The Journal of the Acoustical Society of America.

[29]  Brian C. J. Moore,et al.  Voice pitch as an aid to lipreading , 1981, Nature.

[30]  H Kunov,et al.  Disruptive effects of auditory signal delay on speech perception with lipreading. , 1986, The Journal of auditory research.

[31]  A. Meltzoff,et al.  The bimodal perception of speech in infancy. , 1982, Science.

[32]  L D Braida,et al.  Single Band Amplitude Envelope Cues as an Aid to Speechreading , 1991, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[33]  Kiyoharu Aizawa,et al.  Real-time facial action image synthesis system driven by speech and text , 1990, Other Conferences.

[34]  B Dodd,et al.  Teaching lip-reading: the efficacy of lessons on video. , 1989, British journal of audiology.

[35]  Q Summerfield,et al.  Detection and Resolution of Audio-Visual Incompatibility in the Perception of Vowels , 1984, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[36]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[37]  Allen A. Montgomery,et al.  ANIMAT: A set of programs to generate, edit, and display sequences of vector-based images , 1982 .

[38]  D. Reisberg,et al.  Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. , 1987 .

[39]  R M Farwell.  Speech reading: a research review. , 1976, American annals of the deaf.

[40]  D C Shepherd,et al.  Visual-neural correlate of speechreading ability in normal-hearing adults. , 1977, Journal of speech and hearing research.

[41]  M. Demorest,et al.  Issues and Developments in the Evaluation of Speechreading. , 1988 .

[42]  Kiyoharu Aizawa,et al.  Model-based analysis synthesis image coding (MBASIC) system for a person's face , 1989, Signal Process. Image Commun..

[43]  M. Wertheimer.  Psychomotor Coordination of Auditory and Visual Space at Birth , 1961, Science.

[44]  G P Clarke,et al.  External electrical stimulation of the cochlea: clinical, psychophysical, speech-perceptual and histological findings. , 1979, British journal of audiology.

[45]  V J Samar,et al.  Visual evoked-response components related to speechreading and spatial skills in hearing and hearing-impaired adults. , 1984, Journal of speech and hearing research.

[46]  Q Summerfield,et al.  Use of Visual Information for Phonetic Perception , 1979, Phonetica.

[47]  S. Nishida.  Speech recognition enhancement by lip information , 1986, CHI '86.

[48]  R Conrad,et al.  Lip-reading by deaf and hearing children. , 1977, The British journal of educational psychology.

[49]  Barbara Dodd,et al.  Lip reading in infants: Attention to speech presented in- and out-of-synchrony , 1979, Cognitive Psychology.

[50]  I. Hirsh,et al.  Perceived order in different sense modalities. , 1961, Journal of experimental psychology.

[51]  A. Montgomery,et al.  Physical characteristics of the lips underlying vowel lipreading performance. , 1983, The Journal of the Acoustical Society of America.

[52]  V. Fromkin.  Lip Positions in American English Vowels , 1964 .

[53]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[54]  A. Macleod,et al.  A procedure for measuring auditory and audio-visual speech-reception thresholds for sentences in noise: rationale, evaluation, and recommendations for use. , 1990, British journal of audiology.

[55]  J. L. Miller,et al.  On the role of visual rate information in phonetic perception , 1985, Perception & psychophysics.

[56]  Eric David Petajan,et al.  Automatic Lipreading to Enhance Speech Recognition (Speech Reading) , 1984 .