An imaging system correlating lip shapes with tongue contact patterns for speech pathology research

In this research, an imaging system was built to work with a newly developed electronic device to help people produce sounds correctly. The system consists of two parts, the internal tongue contact pattern data collection and the external lip shape information analysis. The tongue position information was gathered using the palatometer, an innovative tongue contact pattern-tracking device invented by Dr. Samuel Fletcher. The lip shape information was collected by processing images taken from people articulating different sounds. We developed an efficient color image segmentation technique to extract lip contour points and form a closed curve for shape analysis. The geometry invariant turn function vs. normalized length was then calculated for the lip shape for each sound and compared against the turn function of the lips in a resting position to quantify their variations from this reference. Both internal (vocal tract) and external (visible lip shape) information was collected for each of the speech sounds. The lip shape information extracted from the images was then correlated with tongue position information. The test results showed that this imaging system can be used to quantify the lip shape information and its relations with the tongue position and is a potentially useful tool for speech pathology research.

[1]  Longin Jan Latecki,et al.  Application of planar shape comparison to object retrieval in image databases , 2002, Pattern Recognit..

[2]  Alan Wee-Chung Liew,et al.  A new optimization procedure for extracting the point-based lip contour using active shape model , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[3]  Longin Jan Latecki,et al.  Shape Description and Search for Similar Objects in Image Databases , 1999, State-of-the-Art in Content-Based Image and Video Retrieval.

[4]  K. K. Neely.  Effect of Visual Factors on the Intelligibility of Speech , 1956 .

[5]  Eric David Petajan,et al.  Automatic Lipreading to Enhance Speech Recognition (Speech Reading) , 1984 .

[6]  Alice Caplier.  Lip detection and tracking , 2001, Proceedings 11th International Conference on Image Analysis and Processing.

[7]  Tsuhan Chen,et al.  Audio-visual interaction in multimedia communication , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[9]  Esther M. Arkin,et al.  An efficiently computable metric for comparing polygonal shapes , 1991, SODA '90.

[10]  Yochai Konig,et al.  "Eigenlips" for robust speech recognition , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.