Thai sign language translation system using upright speed-up robust feature and c-means clustering

Sign language is an important communication tool for the deaf. In this paper, we build a dynamic hand gesture translation system with video caption without prior hand region segmentation. In particular, we utilize the upright speed-up robust feature (U-SURF) and the fuzzy C-means (FCM) to find a matched word. We compared the result with that from string grammar hard C-means (sgHCM). This Thai sign language translation system was tested on 42 words. The total number of video sequences used in the experiment is 1470. The best correct classification rate on the signer-dependent blind test set using the FCM is approximately 47 to 73%, whereas that of signer-semi-independent set is around 30 to 40%. The best blind test result for the signer-independent experiment is around 24 to 30%. However, the correct classification rate from the sgHCM is higher than that from the FCM. The best result for the signer-dependent experiment is around 97 to 99%, whereas that of the signer-semi-independent is around 64 to 65%. The correct classification rate of the signer-independent experiment is around 53 to 54%.

[1]  直樹 水野,et al.  各指の曲げ状態に着目した手話の手形状認識(機械力学,計測,自動制御) , 2002 .

[2]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[3]  R. Akmeliawati,et al.  Real-Time Malaysian Sign Language Translation using Colour Segmentation and Neural Network , 2007, 2007 IEEE Instrumentation & Measurement Technology Conference IMTC 2007.

[4]  Khaled Assaleh,et al.  Video-based signer-independent Arabic sign language recognition using hidden Markov models , 2009, Appl. Soft Comput..

[5]  Stan Sclaroff,et al.  Simultaneous Localization and Recognition of Dynamic Hand Gestures , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[6]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[7]  Pinit Kumhom,et al.  Hand posture classification using wavelet moment invariant , 2004, 2004 IEEE Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2004. (VCIMS)..

[8]  Chung-Lin Huang,et al.  Sign language recognition using model-based tracking and a 3D Hopfield neural network , 1998, Machine Vision and Applications.

[9]  Ho-Sub Yoon,et al.  Hand gesture recognition using hidden Markov models , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[10]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[11]  Ahmed K. Elmagarmid,et al.  Scene change detection techniques for video database systems , 1998, Multimedia Systems.

[12]  Matthew A. Brown,et al.  Invariant Features from Interest Point Groups , 2002, BMVC.

[13]  Mu-Chun Su,et al.  A static hand gesture recognition system using a composite neural network , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[14]  Yung-Hui Lee,et al.  Taiwan sign language (TSL) recognition based on 3D data and neural networks , 2009, Expert Syst. Appl..

[15]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[16]  Sansanee Auephanwiriyakul,et al.  Thai Sign Language Translation Using Fuzzy C-Means and Scale Invariant Feature Transform , 2008, ICCSA.

[17]  T. Kobayashi,et al.  Partly-hidden Markov model and its application to gesture recognition , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[18]  Wu jiangqin,et al.  A simple sign language recognition system based on data glove , 1998, ICSP '98. 1998 Fourth International Conference on Signal Processing (Cat. No.98TH8344).

[19]  Wen Gao,et al.  A Chinese sign language recognition system based on SOFM/SRN/HMM , 2004, Pattern Recognit..

[20]  Sansanee Auephanwiriyakul,et al.  Incorporating SIFT with Hard C-Means algorithm , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[21]  Quan Yang,et al.  Chinese sign language recognition based on video sequence appearance modeling , 2010, 2010 5th IEEE Conference on Industrial Electronics and Applications.

[22]  James M. Keller,et al.  Fuzzy Models and Algorithms for Pattern Recognition and Image Processing , 1999 .

[23]  Wu Jiang.  A Hierarchical DGMM Recognizer for Chinese Sign Language Recognition , 2000 .