Signer-independent sign language recognition based on SOFM/HMM

The aim of sign language recognition is to provide an efficient and accurate mechanism to transcribe sign language into text or speech. State-of-the-art sign language recognition should be able to solve the signer-independent problem for practical application. In this paper, a hybrid SOFM/HMM system, which combines self-organizing feature maps (SOFMs) with hidden Markov models (HMMs), is presented for signer-independent Chinese sign language recognition. We implement the SOFM/HMM sign recognition system. Meanwhile, results from the HMM-based system are provided as comparison. Experimental results show the SOFM/HMM system increases the recognition accuracy by 5% than the HMM-based one. Furthermore, a self-adjusting recognition algorithm is also proposed for improving the SOFM/HMM discrimination. When it is applied to the SOFM/HMM system it can improve the recognition accuracy by 1.9%. All experiments were performed in real-time with the dictionary size 208.

[1]  Mohammed Waleed Kadous,et al.  Machine Recognition of Auslan Signs Using PowerGloves: Towards Large-Lexicon Recognition of Sign Lan , 1996 .

[2]  Wen Gao,et al.  HandTalker: A Multimodal Dialog System Using Sign Language and 3-D Virtual Human , 2000, ICMI.

[3]  Kirsti Grobel,et al.  Isolated sign language recognition using hidden Markov models , 1996, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[4]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[5]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[6]  Peter Vamplew.  Recognition of sign language gestures using neural networks , 1996 .

[7]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[8]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[9]  A E Marble,et al.  Image processing system for interpreting motion in American Sign Language. , 1992, Journal of biomedical engineering.

[10]  Dimitris N. Metaxas,et al.  Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[11]  Ming Ouhyoung,et al.  A real-time continuous gesture recognition system for sign language , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.