Perception of point light displays of speech by normal-hearing adults and deaf adults with cochlear implants

Normal-hearing (NH) adults display audiovisual enhancement when degraded auditory input (e.g., words, sentences) is paired with point -light displays (PLDs) of speech, which isolate the kinematic properties of a speaker’s face [10]. Do deaf adults who use cochlear implants (CIs) benefit in the same way? Does feedback influence NH adults’ performance? In the present study, we investigated audiovisual (AV) word recognition using PLDs of speech in postlingually deaf adults with CIs and NH adults. Both groups displayed evidence of AV enhancement with PLDs. Moreover, NH participants’ Visual-alone performance improved over time with Auditory-alone and AV feedback. These results suggest that NH and CI adults were sensitive to the kinematic properties in speech represented in the PLDs, and they were able to use kinematics to improve their word recognition performance even with highly degraded visual displays of speech. In addition, NH adults were able to use temporal cues from Auditory-alone and AV feedback to improve their word recognition performance with point -light visual displays of speech.

[1]  L. Bernstein,et al.  Enhanced speechreading in deaf adults: can short-term training/practice close the gap for hearing adults? , 2001, Journal of speech, language, and hearing research : JSLHR.

[2]  Louisa M. Slowiaczek,et al.  An Investigation of Phonology and Orthography in Spoken-Word Recognition , 2003, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[3]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[4]  M. Ross,et al.  Primary modality for speech perception in children with normal and impaired hearing. , 1985, Journal of speech and hearing research.

[5]  B. Walden,et al.  Effects of training on the visual recognition of consonants. , 1977, Journal of speech and hearing research.

[6]  David B Pisoni,et al.  Talker and lexical effects on audiovisual word recognition by adults with cochlear implants. , 2003, Journal of speech, language, and hearing research : JSLHR.

[7]  K. Grant,et al.  Auditory-visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory-visual integration. , 1998, The Journal of the Acoustical Society of America.

[8]  D W Massaro,et al.  Discovery and expository methods in teaching visual consonant and word identification. , 1992, Journal of speech and hearing research.

[9]  J. A. Johnson,et al.  Point-light facial displays enhance comprehension of speech in noise. , 1996, Journal of speech and hearing research.

[10]  N. P. Erber.  Interaction of audition and vision in the recognition of oral speech stimuli. , 1969, Journal of speech and hearing research.

[11]  D B Pisoni,et al.  Use of Audiovisual Information in Speech Perception by Prelingually Deaf Children with Cochlear Implants: A First Report , 2001, Ear and hearing.

[12]  B J Gantz,et al.  Performance over time of adult patients using the Ineraid or nucleus cochlear implant. , 1997, The Journal of the Acoustical Society of America.

[13]  L. Rosenblum,et al.  An audiovisual test of kinematic primitives for visual speech perception. , 1996, Journal of experimental psychology. Human perception and performance.

[14]  G P Bingham,et al.  Dynamics and the orientation of kinematic forms in visual event recognition. , 1995, Journal of experimental psychology. Human perception and performance.

[15]  N. P. Erber.  Auditory-visual perception of speech. , 1975, The Journal of speech and hearing disorders.

[16]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .