Determining the influence of Gaussian blurring on inversion effects with talking faces

Perception of visual speech and the influence of visual speech on auditory speech perception is affected by the orientation of a talker’s face, but the nature of the visual information underlying this effect has yet to be established. Here, we examine the contributions of visually coarse (configural) and fine (featural) facial movement information to inversion effects in the perception of visual and audiovisual speech. We describe two experiments in which we disrupted perception of fine facial detail by decreasing spatial frequency (blurring) and disrupted perception of coarse configural information by facial inversion. For normal, unblurred talking faces, facial inversion had no influence on visual speech identification or on the effects of congruent or incongruent visual speech movements on perception of auditory speech. However, for blurred faces, facial inversion reduced identification of unimodal visual speech and effects of visual speech on perception of congruent and incongruent auditory speech. These effects were more pronounced for words whose appearance may be defined by fine featural detail. Implications for the nature of inversion effects in visual and audiovisual speech are discussed.

[1]  D. Pearson,et al.  Visual Communication Systems for the Deaf , 1981, IEEE Trans. Commun..

[2]  S. Carey,et al.  From piecemeal to configurational representation of faces. , 1977, Science.

[3]  D. Reisberg,et al.  Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. , 1987 .

[4]  T. Bachmann Identification of spatially quantised tachistoscopic images of faces: How many pixels does it take to carry identity? , 1991 .

[5]  J. Bartlett,et al.  Inversion and processing of component and spatial–relational information in faces. , 1996 .

[6]  V. Bruce,et al.  The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology When Inverted Faces Are Recognized: the Role of Configural Information in Face Recognition , 2022 .

[7]  B. Walden,et al.  Effects of training on the visual recognition of consonants. , 1977, Journal of speech and hearing research.

[8]  J. N. Bassili Facial motion in the perception of faces and of emotional expression. , 1978 .

[9]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[10]  Timothy R. Jordan,et al.  Effects of Distance on Visual and Audiovisual Speech Recognition , 2000 .

[11]  D. Berry What can a moving face tell us? , 1990, Journal of personality and social psychology.

[12]  Kerry P. Green,et al.  The influence of an inverted face on the McGurk effect , 1994 .

[13]  H. Wallbott Effects of distortion of spatial and temporal resolution of video stimuli on emotion attributions , 1992 .

[14]  Timothy R. Jordan,et al.  Visual and audiovisual speech perception with color and gray-scale facial images , 2000, Perception & psychophysics.

[15]  D W Massaro,et al.  Perceiving speech from inverted faces , 1996, Perception & psychophysics.

[16]  M. Farah,et al.  Parts and Wholes in Face Recognition , 1993, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[17]  D. Massaro,et al.  Perception of Visible Speech: Influence of Spatial Quantization , 1997, Perception.

[18]  L D Rosenblum,et al.  Face and mouth inversion effects on visual and audiovisual speech perception. , 2000, Journal of experimental psychology. Human perception and performance.

[19]  T. Valentine Upside-down faces: a review of the effect of inversion upon face recognition. , 1988, British journal of psychology.

[20]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[21]  T. R. Jordan,et al.  Seeing and hearing rotated faces: influences of facial orientation on visual and audiovisual speech recognition. , 1997, Journal of experimental psychology. Human perception and performance.

[22]  R. Campbell,et al.  Hearing by eye 2 : advances in the psychology of speechreading and auditory-visual speech , 1997 .

[23]  Takashi Kato,et al.  Influence of facial views on the mcgurk effect in auditory noise , 1998, ICSLP.

[24]  Lawrence D. Rosenblum,et al.  An audiovisual test of kinematic primitives for visual speech perception. , 1996 .

[25]  Béatrice de Gelder,et al.  Exploring the relation between mcgurk interference and ventriloquism , 1994, ICSLP.

[26]  N. P. Erber Interaction of audition and vision in the recognition of oral speech stimuli. , 1969, Journal of speech and hearing research.

[27]  L. Rosenblum,et al.  Discrimination tests of visually influenced syllables , 1992, Perception & psychophysics.

[28]  J. A. Johnson,et al.  Point-light facial displays enhance comprehension of speech in noise. , 1996, Journal of speech and hearing research.

[29]  G. Rhodes Looking at Faces: First-Order and Second-Order Features as Determinants of Facial Appearance , 1988, Perception.

[30]  G. Johansson Visual perception of biological motion and a model for its analysis , 1973 .

[31]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[32]  The contribution of a reduced visual image to speech perception in noise , 1994 .

[33]  Dolf Schinkel,et al.  Improved Speech Recognition Through Videotelephony: Experiments with the Hard of Hearing , 1991, IEEE J. Sel. Areas Commun..

[34]  G. Bingham,et al.  Kinematic form and scaling: further investigations on the visual perception of lifted weight. , 1987, Journal of experimental psychology. Human perception and performance.

[35]  A. Macleod,et al.  A procedure for measuring auditory and audio-visual speech-reception thresholds for sentences in noise: rationale, evaluation, and recommendations for use. , 1990, British journal of audiology.

[36]  R Plomp,et al.  The effect of speechreading on the speech-reception threshold of sentences in noise. , 1987, The Journal of the Acoustical Society of America.

[37]  J. Cutting,et al.  Recognizing the sex of a walker from a dynamic point-light display , 1977 .

[38]  Sharon M. Thomas,et al.  Effects of horizontal viewing angle on visual and audiovisual speech recognition. , 2001, Journal of experimental psychology. Human perception and performance.

[39]  Melanie Vitkovitch,et al.  Visible Speech as a Function of Image Quality: Effects of Display Parameters on Lipreading Ability , 1996 .

[40]  J. Sergent An investigation into component and configural processes underlying face perception. , 1984, British journal of psychology.

[41]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[42]  I. Craw,et al.  Effects of high-pass and low-pass spatial filtering on face identification , 1996, Perception & psychophysics.