Feature adaptation of hearing-impaired lip shapes: the vowel case in the cued speech context

The phonetic translation of Cued Speech (CS) gestures needs to mix the manual CS information together with the lips, taking into account the desynchronization delay (Attina et al. [2], Aboutabit et al. [4]) between these two flows of information. This contribution focuses on the lip flow modeling in the case of French vowels. Previously, classification models have been developed for a professional normal-hearing CS speaker (Aboutabit et al., [7]). These models are used as a reference. In this study, we process the case of a deaf CS speaker and discuss the possibilities of classification. The best performance (92.8%) is obtained with the adaptation of the deaf data to the reference models.

[1]  Denis Beautemps,et al.  A pilot study of temporal organization in Cued Speech production of French syllables: rules for a Cued Speech synthesizer , 2004, Speech Commun..

[2]  Douglas D. O'Shaughnessy,et al.  A new approach for phoneme segmentation of speech signals , 2007, INTERSPEECH.

[3]  G. H. Nicholls,et al.  Cued Speech and the reception of spoken language. , 1982, Journal of speech and hearing research.

[4]  Noureddine Aboutabit,et al.  Analyse de la production d'un codeur LPC sourd. , 2008 .

[5]  Gérard Bailly,et al.  Talking Machines: Theories, Models, and Designs , 1992 .

[6]  A. Montgomery,et al.  Physical characteristics of the lips underlying vowel lipreading performance. , 1983, The Journal of the Acoustical Society of America.

[7]  L. Bernstein,et al.  Speech perception without hearing , 2000, Perception & psychophysics.

[8]  J. Leybaert,et al.  Phonology acquired through the eyes and spelling in deaf children. , 2000, Journal of experimental child psychology.

[9]  Gérard Bailly,et al.  TELMA : Telephony for the Hearing-Impaired People. From Models to User Tests , 2007 .

[10]  Dominique Vaufreydaz,et al.  A New Methodology for Speech Corpora Definition from Internet Documents , 2000, LREC.

[11]  Denis Beautemps,et al.  Hand and Lip Desynchronization Analysis in French Cued Speech: Automatic Temporal Segmentation of Hand Flow , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[12]  C M Reed,et al.  Automatic speech recognition to aid the hearing impaired: prospects for the automatic generation of cued speech. , 1994, Journal of rehabilitation research and development.

[13]  C. Benoît,et al.  A set of French visemes for visual speech synthesis , 1994 .

[14]  M. C. Jones.  Cued speech. , 1992, ASHA.