Increasing the Effect of Fingers in Fingerspelling Hand Shapes by Thick Edge Detection and Correlation with Penalization

Fingerspelling is used in sign language to spell out names of people and places for which there is no sign or for which the sign is not known. In this work we describe a method for increasing the effect of fingers in Fingerspelling hand shapes. Hand shape objects are obtained by extraction of representative frames, color segmentation in YCrCb space and angle of least inertia based fast alignment [1]. Thick edges of the hand shape objects are extracted with a distance to edge based method. Finally a calculation that penalizes similarity for not-corresponding pixels is employed to correlation based template matching. The experimental Turkish fingerspelling recognition system recognizes all 29 letters of the Turkish alphabet. The train video database is created by three signers, and has a set of 290 videos. The test video database is created by four signers, and has a set of 203 videos. Our methods achieve a success rate of 99%.

[1]  Akira Iwata,et al.  Hand alphabet recognition using morphological PCA and neural networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[2]  Oguz Altun,et al.  Turkish Fingerspelling Recognition System Using Axis of Least Inertia Based Fast Alignment , 2006, Australian Conference on Artificial Intelligence.

[3]  Alex Pentland,et al.  Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Ramesh Raskar,et al.  Exploiting Depth Discontinuities for Vision-Based Fingerspelling Recognition , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[5]  Songül Albayrak,et al.  Real Time Isolated Turkish Sign Language Recognition from Video Using Hidden Markov Models with Global Features , 2005, ISCIS.

[6]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[7]  Wen Gao,et al.  A Chinese sign language recognition system based on SOFM/SRN/HMM , 2004, Pattern Recognit..

[8]  S. Foo,et al.  Hand pose estimation for American sign language recognition , 2004, Thirty-Sixth Southeastern Symposium on System Theory, 2004. Proceedings of the.

[9]  Songül Albayrak,et al.  A Two-Stage Visual Turkish Sign Language Recognition System Based on Global and Local Features , 2006, ISMIS.

[10]  Robyn A. Owens,et al.  Australian sign language recognition , 2005, Machine Vision and Applications.

[11]  Robert W. Lindeman,et al.  A multi-class pattern recognition system for practical finger spelling translation , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.