The role of visual speech cues in reducing energetic and informational masking.

Two experiments compared the effect of supplying visual speech information (e.g., lipreading cues) on the ability to hear one female talker's voice in the presence of steady-state noise or a masking complex consisting of two other female voices. In the first experiment intelligibility of sentences was measured in the presence of the two types of maskers with and without perceived spatial separation of target and masker. The second study tested detection of sentences in the same experimental conditions. Results showed that visual cues provided more benefit for both recognition and detection of speech when the masker consisted of other voices (versus steady-state noise). Moreover, visual cues provided greater benefit when the target speech and masker were spatially coincident versus when they appeared to arise from different spatial locations. The data obtained here are consistent with the hypothesis that lipreading cues help to segregate a target voice from competing voices, in addition to the established benefit of supplementing masked phonetic information.

[1]  Ira J. Hirsh,et al.  The Relation between Localization and Intelligibility , 1950 .

[2]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[3]  T W Tillman,et al.  Perceptual masking in multiple sound backgrounds. , 1969, The Journal of the Acoustical Society of America.

[4]  R R Coles,et al.  Binaural advantages in hearing of speech , 1971, The Journal of Laryngology & Otology.

[5]  H. Levitt.  Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[6]  D. Reisberg,et al.  Looking where you listen: visual cues and auditory attention. , 1978, Acta psychologica.

[7]  Q Summerfield,et al.  Use of Visual Information for Phonetic Perception , 1979, Phonetica.

[8]  D. Reisberg,et al.  Eye position and the control of auditory attention. , 1981, Journal of experimental psychology. Human perception and performance.

[9]  R Plomp,et al.  Effect of multiple speechlike maskers on binaural speech recognition in normal and impaired hearing. , 1992, The Journal of the Acoustical Society of America.

[10]  S P Bacon,et al.  Modulation detection, modulation masking, and speech understanding in noise in the elderly. , 1992, Journal of speech and hearing research.

[11]  Elizabeth C. Zsiga,et al.  Lexical Mediation between Sight and Sound in Speechreading , 1992, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[12]  P. Barber,et al.  Effect of video frame rate on subjects' ability to shadow one of two competing verbal passages. , 1994, Journal of speech and hearing research.

[13]  Ken W. Grant,et al.  The recognition of isolated words and words in sentences: Individual variability in the use of sentence context , 1997 .

[14]  K S Helfer,et al.  Auditory and auditory-visual perception of clear and conversational speech. , 1997, Journal of speech, language, and hearing research : JSLHR.

[15]  C. Mason,et al.  Release from masking due to spatial separation of sources in the identification of nonspeech auditory patterns. , 1998, The Journal of the Acoustical Society of America.

[16]  R L Freyman,et al.  The role of perceived spatial separation in the unmasking of speech. , 1999, The Journal of the Acoustical Society of America.

[17]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[18]  J Driver,et al.  Cross-modal selective attention: On the difficulty of ignoring sounds at the locus of visual attention , 2000, Perception & psychophysics.

[19]  P F Seitz,et al.  The use of visible speech cues for improving auditory detection of spoken sentences. , 2000, The Journal of the Acoustical Society of America.

[20]  K. Grant,et al.  The effect of speechreading on masked detection thresholds for filtered speech. , 2001, The Journal of the Acoustical Society of America.

[21]  R L Freyman,et al.  Spatial release from informational masking in speech recognition. , 2001, The Journal of the Acoustical Society of America.

[22]  D S Brungart,et al.  Informational and energetic masking effects in the perception of two simultaneous talkers. , 2001, The Journal of the Acoustical Society of America.

[23]  G. Kidd,et al.  The effect of spatial separation on informational and energetic masking of speech. , 2002, The Journal of the Acoustical Society of America.

[24]  William Noble,et al.  Hearing speech against spatially separate competing speech versus competing noise , 2002, Perception & psychophysics.

[25]  Jean-Pierre Gagné,et al.  Auditory, visual and audiovisual clear speech , 2002, Speech Commun..

[26]  Jason S. McCarley,et al.  Bimodal Displays Improve Speech Comprehension in Environments with Multiple Speakers , 2003, Hum. Factors.

[27]  Richard L Freyman,et al.  Effect of number of masking talkers and auditory priming on informational masking in speech recognition. , 2004, The Journal of the Acoustical Society of America.