The application of extension neuro-network on computer-assisted lip-reading recognition for hearing impaired

Within the communication process of human beings, the speaker's facial expression and lip-shape movement contains extremely rich language information. The hearing impaired, aside from using residual listening to communicate with other people, can also use lip reading as a communication tool. As the hearing impaired learn the lip reading using a computer-assisted lip-reading system, they can freely learn lip reading without the constraints of time, place or situation. Therefore, we propose a computer-assisted lip-reading system (CALRS) for phonetic pronunciation recognition of the correct lip-shape with an image processing method, object-oriented language and neuro-network. This system can accurately compare the lip image of Mandarin phonetic pronunciation using self-organizing map neuro-network (SOMNN) and extension theory to help hearing impaired correct their pronunciation.

[1]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[2]  Shenghua Ye,et al.  Fully automatic algorithm for region of interest location in camera calibration , 2002 .

[3]  Ioannis Hatzilygeroudis,et al.  Integrating (rules, neural networks) and cases for knowledge representation and reasoning in expert systems , 2004, Expert Syst. Appl..

[4]  Yu Yongquan,et al.  The Research of Data Mining Based on Extension Sets , 2005, Third International Conference on Information Technology and Applications (ICITA'05).

[5]  K P Green,et al.  Cross-modal discrepancies in coarticulation and the integration of speech information: the McGurk effect with mismatched vowels. , 1995, Journal of experimental psychology. Human perception and performance.

[6]  Yun-Long Lay,et al.  Implementation and evaluation of computer-aided Mandarin phonemes training system for hearing-impaired students , 2005, Br. J. Educ. Technol..

[7]  Yu Yongquan,et al.  Extension set and restricting qualifications of matter-elements' extension , 2005, Third International Conference on Information Technology and Applications (ICITA'05).

[8]  Mang-Hui Wang,et al.  Application of extension theory to PD pattern recognition in high-voltage current transformers , 2005 .

[9]  M. H. Wang,et al.  A Novel Extension Method for Transformer Fault Diagnosis , 2002, IEEE Power Engineering Review.

[10]  Shu Hung Leung,et al.  A real-time automatic lipreading system , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[11]  B. Klinteberg,et al.  Personality and work life: A comparison between hearing impaired persons and a normal hearing population : Social Behavior and Personality , 2003 .